-- Logs begin at Wed 2022-07-20 14:04:19 UTC, end at Wed 2022-07-20 15:39:01 UTC. -- Jul 20 14:33:05.926491 dv-cinder-849022-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 20 14:33:05.939106 dv-cinder-849022-4-iscsi devstack@keystone.service[86850]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 20 14:33:05.939659 dv-cinder-849022-4-iscsi devstack@keystone.service[86850]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jul 20 14:33:05 2022] *** Jul 20 14:33:05.939659 dv-cinder-849022-4-iscsi devstack@keystone.service[86850]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 20 14:33:05.939659 dv-cinder-849022-4-iscsi devstack@keystone.service[86850]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 20 14:33:05.939659 dv-cinder-849022-4-iscsi devstack@keystone.service[86850]: nodename: dv-cinder-849022-4-iscsi Jul 20 14:33:05.939659 dv-cinder-849022-4-iscsi devstack@keystone.service[86850]: machine: x86_64 Jul 20 14:33:05.939659 dv-cinder-849022-4-iscsi devstack@keystone.service[86850]: clock source: unix Jul 20 14:33:05.939659 dv-cinder-849022-4-iscsi devstack@keystone.service[86850]: pcre jit disabled Jul 20 14:33:05.939659 dv-cinder-849022-4-iscsi devstack@keystone.service[86850]: detected number of CPU cores: 4 Jul 20 14:33:05.939659 dv-cinder-849022-4-iscsi devstack@keystone.service[86850]: current working directory: / Jul 20 14:33:05.939659 dv-cinder-849022-4-iscsi devstack@keystone.service[86850]: detected binary path: /usr/bin/uwsgi-core Jul 20 14:33:05.939659 dv-cinder-849022-4-iscsi devstack@keystone.service[86850]: your processes number limit is 47680 Jul 20 14:33:05.939659 dv-cinder-849022-4-iscsi devstack@keystone.service[86850]: your memory page size is 4096 bytes Jul 20 14:33:05.939659 dv-cinder-849022-4-iscsi devstack@keystone.service[86850]: detected max file descriptor number: 2048 Jul 20 14:33:05.939659 dv-cinder-849022-4-iscsi devstack@keystone.service[86850]: lock engine: pthread robust mutexes Jul 20 14:33:05.939659 dv-cinder-849022-4-iscsi devstack@keystone.service[86850]: thunder lock: enabled Jul 20 14:33:05.939659 dv-cinder-849022-4-iscsi devstack@keystone.service[86850]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 20 14:33:05.939659 dv-cinder-849022-4-iscsi devstack@keystone.service[86850]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 20 14:33:05.983160 dv-cinder-849022-4-iscsi devstack@keystone.service[86850]: Python main interpreter initialized at 0x562513afd9c0 Jul 20 14:33:05.983160 dv-cinder-849022-4-iscsi devstack@keystone.service[86850]: python threads support enabled Jul 20 14:33:05.983160 dv-cinder-849022-4-iscsi devstack@keystone.service[86850]: your server socket listen backlog is limited to 100 connections Jul 20 14:33:05.983160 dv-cinder-849022-4-iscsi devstack@keystone.service[86850]: your mercy for graceful operations on workers is 90 seconds Jul 20 14:33:05.983559 dv-cinder-849022-4-iscsi devstack@keystone.service[86850]: mapped 403077 bytes (393 KB) for 2 cores Jul 20 14:33:05.983559 dv-cinder-849022-4-iscsi devstack@keystone.service[86850]: *** Operational MODE: preforking *** Jul 20 14:33:05.983892 dv-cinder-849022-4-iscsi devstack@keystone.service[86850]: *** uWSGI is running in multiple interpreter mode *** Jul 20 14:33:05.983892 dv-cinder-849022-4-iscsi devstack@keystone.service[86850]: spawned uWSGI master process (pid: 86850) Jul 20 14:33:05.984126 dv-cinder-849022-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 20 14:33:05.984671 dv-cinder-849022-4-iscsi devstack@keystone.service[86850]: spawned uWSGI worker 1 (pid: 86860, cores: 1) Jul 20 14:33:05.985423 dv-cinder-849022-4-iscsi devstack@keystone.service[86850]: spawned uWSGI worker 2 (pid: 86861, cores: 1) Jul 20 14:33:05.985667 dv-cinder-849022-4-iscsi devstack@keystone.service[86850]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 20 14:33:11.332098 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 14:33:11.332098 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: from cryptography.utils import int_from_bytes Jul 20 14:33:11.332956 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 14:33:11.332956 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: from cryptography.utils import int_from_bytes Jul 20 14:33:11.333060 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 14:33:11.333060 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: from cryptography.utils import int_from_bytes Jul 20 14:33:11.333060 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 14:33:11.333060 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: from cryptography.utils import int_from_bytes Jul 20 14:33:11.381259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 20 14:33:11.381496 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 20 14:33:11.381646 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] command line args: [] {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 20 14:33:11.381796 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 20 14:33:11.381885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 20 14:33:11.381966 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 20 14:33:11.382047 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] command line args: [] {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 20 14:33:11.382126 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ================================================================================ {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 20 14:33:11.382321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 20 14:33:11.382455 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] admin_token = **** {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.382535 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ================================================================================ {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 20 14:33:11.382602 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] config_dir = [] {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.382799 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.382924 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] admin_token = **** {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.383014 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] config_source = [] {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.383118 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] config_dir = [] {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.383206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] debug = True {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.383291 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.383371 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.384724 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] config_source = [] {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.384724 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] debug = True {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.384855 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.384927 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.385012 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.385079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.385143 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.385202 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.385267 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.385332 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.385395 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] list_limit = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.385458 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.385516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] log_config_append = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.385573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] list_limit = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.385638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.385698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] log_config_append = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.385755 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] log_dir = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.385755 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] log_file = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.385863 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.385863 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] log_dir = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.385973 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.386037 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] log_file = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.386096 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.386152 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.386215 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.386278 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.386344 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.386411 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.386470 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.386470 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.386587 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.386587 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.386708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.386779 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.386858 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.386923 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.386982 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.387455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.387560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.387659 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.387735 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.387861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.387977 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.388100 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.388213 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.388292 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.388352 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.388417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.388475 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.388546 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.388858 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.388991 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.389145 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] publish_errors = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.389328 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.389450 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.389532 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] publish_errors = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.389676 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.389809 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.389888 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.389966 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.390045 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.390118 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.390233 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.390316 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.390530 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] standard_threads = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.390655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.390745 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.390820 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] standard_threads = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.391027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.391171 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.391268 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.391343 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.391415 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] use_journal = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.391608 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.391692 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] use_json = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.391765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] use_journal = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.391842 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] use_stderr = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.391914 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] use_json = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.391914 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] use_stderr = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.392052 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] use_syslog = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.392124 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] use_syslog = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.392201 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.392323 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:33:11.392595 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.392729 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.392828 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.392905 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.393084 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.393241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.393343 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.393549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.393686 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.393768 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.393944 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.394069 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.394204 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.394295 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.394482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.394604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.394684 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.394882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.395006 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.395214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.395356 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.395440 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.395551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.395640 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.395931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.396054 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.396152 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.396235 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.396401 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.396525 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.396638 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.396734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.396898 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.396981 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.397133 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.397217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.397370 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.397562 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.397682 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.397767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.397960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.398050 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.398220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.398304 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.398470 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.398554 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.398705 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.398838 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.398927 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.399143 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.399244 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.399405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.399487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.399657 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.399740 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.399902 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.399992 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.400135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.400219 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.400365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.400456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.400613 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.400697 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.400860 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.400944 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.401100 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.401184 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.401348 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.401432 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.401563 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.401648 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.401799 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.401925 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.402037 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.402203 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.402287 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.402466 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.402545 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.402705 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.402860 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.402937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.403008 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.403186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.403293 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.403435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.403556 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.403667 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.403894 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.403972 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.404122 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.404201 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.404349 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.404434 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.404577 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.404656 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.404812 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.404918 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.405024 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.405145 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.405275 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.405375 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.405479 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.405584 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.405690 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.405792 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.405923 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.406009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.406155 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.406264 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.406396 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.406481 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.406623 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.406700 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.406855 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.406934 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.407106 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.407190 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.407343 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.407438 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.407574 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.407652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.407807 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.407896 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.408033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.408120 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.408272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.408350 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.408502 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.408589 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.408738 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.408816 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.408965 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.409052 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.409219 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.409304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.409452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.409532 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.409678 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.409752 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.409909 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.409987 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.410146 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.410231 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.410393 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.410470 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.410633 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.410721 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.410873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.410952 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.411134 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.411215 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.411371 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.411465 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.411604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.411680 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.411837 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.411916 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.412076 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.412162 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.412309 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.412388 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.412542 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.412622 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.412780 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.412871 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.413042 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.413121 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.413277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.413378 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.413508 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.413626 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.413740 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.413867 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.413971 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.414107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.414229 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.414387 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.414463 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.414565 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.414690 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.414797 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.414925 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.415033 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.415186 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.415278 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.415422 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.415507 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.415659 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.415739 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.415893 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.415980 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.416121 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.416201 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.416348 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.416428 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.416577 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.416689 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.416808 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.416908 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.417019 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.417263 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.417406 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.417545 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.417688 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] auth.external = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.417769 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.417939 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.418104 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.418198 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.418444 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.418659 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.418741 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.418894 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] auth.external = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.419113 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.419233 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.419424 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.419673 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] auth.password = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.419756 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.419883 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] auth.password = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.420012 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] auth.token = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.420158 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] auth.token = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.420244 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.420330 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.420406 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.420520 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.420663 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.420743 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.420908 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.420998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.421166 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.421247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.421429 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.421509 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.421657 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.421744 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.421888 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] credential.caching = True {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.421969 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] credential.caching = True {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.422130 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.422211 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.422375 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.422464 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.422617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.422698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.422848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.422928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.423108 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.423204 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.423340 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.423420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.423600 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.423679 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.423848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.423937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.424079 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.424160 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.424671 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.424758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.424962 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.425051 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.425207 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.425287 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.425474 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.425561 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.425717 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.425804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.426024 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.426111 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.426186 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] federation.caching = True {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.426258 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] federation.caching = True {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.426415 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.426505 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.426659 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.426738 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.426902 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.426989 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.427151 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.427247 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.427395 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.427483 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.427627 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.427707 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.427874 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.427963 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.428106 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.428186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.428360 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.428439 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.428590 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.428678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.428905 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.428987 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.429142 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] identity.caching = True {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.429220 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] identity.caching = True {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.429384 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.429546 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.429623 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.429697 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.429846 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.429927 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.430077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.430159 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.430450 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.430531 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.430681 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.430762 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.430918 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.431000 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.431189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.431271 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.431441 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.431522 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.431674 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.431756 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.431904 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.431984 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.432132 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.432212 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.432379 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.432497 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.432620 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.432693 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.432873 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.433029 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.433176 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.433251 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.433347 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.433431 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.433613 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.433693 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.433853 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.433935 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.434099 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.434181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.434320 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.434400 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.434552 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.434634 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.434780 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.434867 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.435010 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.435153 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.435273 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.435355 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.435509 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.435596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.435753 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.435833 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.435982 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.436064 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.436225 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.436305 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.436468 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.436549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.436701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.436782 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.436943 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.437023 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.437185 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.437265 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.437412 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.437494 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.437645 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.437726 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.437872 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.437952 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.438104 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.438185 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.438341 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.438427 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.438580 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.438659 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.438814 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.438896 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.439064 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.439147 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.439307 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.439387 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.439550 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.439632 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.439776 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.439857 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.440002 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.440082 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.440245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.440326 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.440485 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.440565 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.440715 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.440795 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.440945 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.441027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.441173 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.441253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.441400 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.user = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.441480 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.user = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.441637 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.441718 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.441892 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.441973 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.442119 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.442207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.442362 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.442442 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.442603 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.442683 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.442843 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.442931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.443095 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.443179 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.443325 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.443405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.443555 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.443648 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.443790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.443869 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.444023 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.444102 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.444272 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.444361 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.444516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.444596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.444757 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.444837 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.444997 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.445086 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.445237 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.445318 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.445477 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.445557 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.445703 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.445791 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.445943 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.446023 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.446176 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.446256 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.446407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.446497 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.446637 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.446718 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.446867 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.446947 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.447131 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.447227 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.447377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.447458 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.447608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.447689 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.447859 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.447981 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.448098 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.448188 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.448316 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.448396 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.448546 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.448631 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.448775 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.448846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.449005 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.449076 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.449249 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.449329 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.449485 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.449555 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.449712 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.449877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.449953 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.450025 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.450167 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] resource.caching = True {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.450237 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] resource.caching = True {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.450409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.450479 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.450650 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.450739 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.450881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.450969 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.451149 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.451233 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.451396 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.451490 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.451626 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.451706 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.451867 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.451947 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.452098 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.452260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.452337 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.452410 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.452550 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] role.caching = True {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.452630 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] role.caching = True {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.452789 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] role.driver = sql {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.452877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] role.driver = sql {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.453013 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.453094 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.453258 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.453338 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.453500 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.453590 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.453741 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.453821 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.453983 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.454063 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.454223 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.454304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.454463 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.454543 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.454702 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.454789 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.454943 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.455024 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.455194 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.455276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.455436 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.455516 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.455678 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.455760 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.455918 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.456003 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.456157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.456238 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.456577 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.456659 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.456806 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.456886 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.457050 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.457146 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.457293 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.457374 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.457535 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.457615 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.457776 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.457856 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.458004 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.458087 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.458256 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.458336 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.458502 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.458582 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.458738 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.458821 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.458967 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.459066 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.459222 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.459302 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.459452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.459533 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.459699 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.459779 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.460004 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.460093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.460249 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.460329 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.460482 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.460562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.460712 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.460801 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.460939 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.461019 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.461169 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] token.caching = True {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.461248 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] token.caching = True {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.461401 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.461490 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.461641 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.461721 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.461875 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.461955 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.462126 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.462215 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.462368 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.462448 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.462601 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.462682 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.462839 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.462928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.463122 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.463208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.463337 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.463416 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.463577 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.463714 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.463819 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.463897 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.464029 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.464109 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.464269 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.464356 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.464511 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.464593 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.464738 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.464818 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.464978 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.465067 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.465266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.465345 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.465520 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] database.connection = **** {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.465609 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] database.connection = **** {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.465758 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.465849 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.465998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.466079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.466232 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.466313 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.466467 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.466548 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.466698 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.466785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.466934 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.467016 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.467185 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.467267 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.467416 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.467496 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.467652 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.467734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.467882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.467963 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.468125 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.468205 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.468354 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.468435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.468595 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.468676 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.468826 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.468906 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.469063 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.469145 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.469288 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.469368 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.469522 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.469602 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.469752 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.469833 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:33:11.469910 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86860) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 20 14:33:11.469983 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 20 14:33:11.501682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86860) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.501874 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.501962 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86860) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.502043 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.502117 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86860) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.502193 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.510489 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86860) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.510656 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.539284 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86860) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.539498 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86860) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.541783 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.542104 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.686016 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.686699 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86860) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.735573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.735814 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.736027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86860) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.736263 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86860) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.801022 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86860) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.801228 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.801300 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86860) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.801362 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.801463 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86860) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.801542 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.801604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86860) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.801664 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.801848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86860) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.801987 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.802057 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86860) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.802126 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.827987 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86860) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.828195 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.828291 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86860) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.828364 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.828439 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86860) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.828501 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.828605 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86860) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.828705 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.828809 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86860) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.828898 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.829063 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86860) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.829200 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.829271 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86860) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.829367 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.841013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.841191 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86860) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.854480 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86860) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.854736 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86860) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.854938 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86860) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.855189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86860) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.855399 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86860) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.855589 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86860) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.855831 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86860) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.857012 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.857223 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.857491 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.857689 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.857942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.858158 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.858376 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 14:33:11.873348 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.873512 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 14:33:11.873578 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.873767 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 14:33:11.873924 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.874055 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 14:33:11.874148 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.874294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 14:33:11.874416 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.874600 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.874745 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.874817 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.874892 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.875099 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.875224 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.875384 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.886295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.886630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.886753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.887385 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.887479 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.887580 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.887659 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.887659 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.895232 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86860) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:11.895570 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:11.898511 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86860) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:11.898806 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.898929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:11.899081 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.899178 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.899472 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.899594 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.899792 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.899926 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.900009 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.900094 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.900306 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.900431 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.900554 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.900645 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.900845 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.900983 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.901066 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.901140 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.901365 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.901491 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.901673 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.928207 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.929067 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.929844 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86860) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:11.930117 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.930690 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:11.930972 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.935357 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86860) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:11.935638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.935918 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.936322 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:11.936542 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.936844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.943147 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86860) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:11.943427 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.944103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:11.944455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.948090 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86860) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:11.948375 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.948650 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.948924 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.949099 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:11.949230 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.949408 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.949535 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.949701 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.949828 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.949924 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.950002 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.950227 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.950502 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.950765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.951072 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.968528 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.968816 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.969821 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.970085 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.973341 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86860) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:11.975438 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:11.976986 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.977276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.979198 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.979494 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.982272 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86860) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:11.984490 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:11.985931 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86860) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:11.988088 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:11.989666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.989965 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.990253 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.990544 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.990825 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.991130 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.991872 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.992175 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.992463 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.992761 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.993049 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:11.993326 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.010273 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86860) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:12.010645 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.010920 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.011197 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.012383 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:12.012763 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.013040 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.013310 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.018988 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.020710 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.020978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.021173 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.022979 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.023279 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.024101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86860) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:12.024379 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.024656 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.024933 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.025211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.026731 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:12.026731 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.026926 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.027183 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.027464 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.038467 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86860) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:12.038740 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.039018 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.039313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.039587 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.039873 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.040153 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.040885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:12.041170 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.041453 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.041733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.041999 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.042291 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.042565 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.059686 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86860) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:12.062304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:12.062922 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86860) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:12.065603 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:12.066198 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.067818 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.068926 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.069456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86860) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:12.069736 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.070024 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.070557 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.072218 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:12.072501 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.072791 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.077361 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.078964 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86860) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:12.080269 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.081886 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:12.082211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.082468 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.082773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.083066 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.085162 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.085452 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.085749 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.086025 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.092327 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86860) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:12.092617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.092894 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.095282 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:12.095581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.095858 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.100721 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86860) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:12.100999 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.101280 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.101551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.101816 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.102106 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.102352 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.102631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.102882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.103189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.103480 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.103735 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.103828 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 14:33:12.104014 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.104142 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.104224 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86860) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.104320 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.104578 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.104854 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.105133 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.105416 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.105704 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.105977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.106243 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.106520 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.106771 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.107035 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.107332 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 14:33:12.141985 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x562513afd9c0 pid: 86860 (default app) Jul 20 14:33:12.145549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f1c16-2483-4527-ad8d-ddcba0c99fa5 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:12.145836 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f1c16-2483-4527-ad8d-ddcba0c99fa5 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:12.146094 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f1c16-2483-4527-ad8d-ddcba0c99fa5 None None] PATH_INFO: `/v3` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:12.147331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x562513afd9c0 pid: 86861 (default app) Jul 20 14:33:12.150471 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1/1] 10.222.0.15 () {50 vars in 791 bytes} [Wed Jul 20 14:33:12 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:19.762951 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46cc92-65a6-4e92-ba40-0c919371ddaf None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:19.763541 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46cc92-65a6-4e92-ba40-0c919371ddaf None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:19.763541 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46cc92-65a6-4e92-ba40-0c919371ddaf None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:19.768825 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1/2] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:19 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:19.774789 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbd890-9221-4c1c-bc47-2a1a9ce61e54 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:19.775323 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbd890-9221-4c1c-bc47-2a1a9ce61e54 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:19.775323 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbd890-9221-4c1c-bc47-2a1a9ce61e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:19.844152 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: INFO dbcounter [None req-7bbbd890-9221-4c1c-bc47-2a1a9ce61e54 None None] Registered counter for database keystone Jul 20 14:33:19.866142 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG oslo_db.sqlalchemy.engines [None req-7bbbd890-9221-4c1c-bc47-2a1a9ce61e54 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86860) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 14:33:19.873160 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writer thread running {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 14:33:20.052374 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG passlib.handlers.bcrypt [None req-7bbbd890-9221-4c1c-bc47-2a1a9ce61e54 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86860) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 20 14:33:20.052757 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG passlib.handlers.bcrypt [None req-7bbbd890-9221-4c1c-bc47-2a1a9ce61e54 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86860) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 20 14:33:20.108039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-7bbbd890-9221-4c1c-bc47-2a1a9ce61e54 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:20.109699 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-7bbbd890-9221-4c1c-bc47-2a1a9ce61e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:20.240079 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 2/3] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:19 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 466 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 20 14:33:20.458901 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46cc92-65a6-4e92-ba40-0c919371ddaf None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:20.533961 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: INFO dbcounter [None req-8f46cc92-65a6-4e92-ba40-0c919371ddaf None None] Registered counter for database keystone Jul 20 14:33:20.554754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG oslo_db.sqlalchemy.engines [None req-8f46cc92-65a6-4e92-ba40-0c919371ddaf None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86861) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 14:33:20.561874 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writer thread running {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 14:33:20.698324 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46cc92-65a6-4e92-ba40-0c919371ddaf None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:20.703253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5600c-64d6-446d-98d6-277d6a470f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:20.703939 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5600c-64d6-446d-98d6-277d6a470f10 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:20.704224 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5600c-64d6-446d-98d6-277d6a470f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:20.704502 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5600c-64d6-446d-98d6-277d6a470f10 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:20.713702 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a5600c-64d6-446d-98d6-277d6a470f10 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:20.765129 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c6a5600c-64d6-446d-98d6-277d6a470f10 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 20 14:33:20.765771 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a5600c-64d6-446d-98d6-277d6a470f10 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:20.776086 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 2/4] 10.222.0.15 () {58 vars in 1324 bytes} [Wed Jul 20 14:33:20 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 14:33:20.782531 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbbd890-9221-4c1c-bc47-2a1a9ce61e54 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:20.798079 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbbd890-9221-4c1c-bc47-2a1a9ce61e54 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:20.803926 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a77b6-fd25-4e09-8b6b-67435250a890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:20.805082 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a77b6-fd25-4e09-8b6b-67435250a890 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:20.805378 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a77b6-fd25-4e09-8b6b-67435250a890 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:20.805657 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a77b6-fd25-4e09-8b6b-67435250a890 admin admin] PATH_INFO: `/v3/projects/3ca83facd1c8420dacb9e91e8abe0fd4` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:20.809959 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a77b6-fd25-4e09-8b6b-67435250a890 admin admin] RBAC: Authorizing `identity:get_project(project_id=3ca83facd1c8420dacb9e91e8abe0fd4)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:20.861598 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: WARNING keystone.common.rbac_enforcer.enforcer [None req-123a77b6-fd25-4e09-8b6b-67435250a890 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 20 14:33:20.861960 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a77b6-fd25-4e09-8b6b-67435250a890 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:20.867275 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 3/5] 10.222.0.15 () {56 vars in 1207 bytes} [Wed Jul 20 14:33:20 2022] GET /identity/v3/projects/3ca83facd1c8420dacb9e91e8abe0fd4 => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:22.914548 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcbde0-60a0-4b14-9b40-cdb358c60e0c None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:22.915458 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcbde0-60a0-4b14-9b40-cdb358c60e0c None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:22.915458 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcbde0-60a0-4b14-9b40-cdb358c60e0c None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:22.922484 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 3/6] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:22 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:22.929970 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17564a7-4149-4f15-a9a0-fb6a3a7320f8 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:22.930390 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17564a7-4149-4f15-a9a0-fb6a3a7320f8 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:22.930579 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17564a7-4149-4f15-a9a0-fb6a3a7320f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:22.955770 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-f17564a7-4149-4f15-a9a0-fb6a3a7320f8 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:22.957298 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-f17564a7-4149-4f15-a9a0-fb6a3a7320f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:22.967449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 4/7] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 20 14:33:22.989732 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcbde0-60a0-4b14-9b40-cdb358c60e0c None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:23.005395 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcbde0-60a0-4b14-9b40-cdb358c60e0c None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:23.010554 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c439587-a7ac-4492-9e03-8ba7ef8f8745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:23.011215 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c439587-a7ac-4492-9e03-8ba7ef8f8745 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:23.011489 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c439587-a7ac-4492-9e03-8ba7ef8f8745 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:23.011769 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c439587-a7ac-4492-9e03-8ba7ef8f8745 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:23.019733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c439587-a7ac-4492-9e03-8ba7ef8f8745 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:23.021535 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c439587-a7ac-4492-9e03-8ba7ef8f8745 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:23.030833 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 4/8] 10.222.0.15 () {58 vars in 1324 bytes} [Wed Jul 20 14:33:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 14:33:23.036540 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17564a7-4149-4f15-a9a0-fb6a3a7320f8 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:23.046136 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17564a7-4149-4f15-a9a0-fb6a3a7320f8 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:23.052106 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5952ac00-3878-48af-b600-c471ace2084d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:23.052990 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952ac00-3878-48af-b600-c471ace2084d admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:23.053273 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952ac00-3878-48af-b600-c471ace2084d admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:23.053543 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952ac00-3878-48af-b600-c471ace2084d admin admin] PATH_INFO: `/v3/users/651b2280231048d59d156e569aa33194` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:23.055092 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5952ac00-3878-48af-b600-c471ace2084d admin admin] RBAC: Authorizing `identity:get_user(user_id=651b2280231048d59d156e569aa33194)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:23.056948 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5952ac00-3878-48af-b600-c471ace2084d admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:23.058552 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 5/9] 10.222.0.15 () {56 vars in 1198 bytes} [Wed Jul 20 14:33:23 2022] GET /identity/v3/users/651b2280231048d59d156e569aa33194 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:25.145366 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16a6c6-5a91-40a4-a359-c31d81459e68 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:25.145915 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16a6c6-5a91-40a4-a359-c31d81459e68 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:25.145915 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16a6c6-5a91-40a4-a359-c31d81459e68 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:25.147481 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 5/10] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:25.152733 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197cce2-dffc-4bdb-b321-81e3cff1dc88 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:25.153183 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197cce2-dffc-4bdb-b321-81e3cff1dc88 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:25.153271 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197cce2-dffc-4bdb-b321-81e3cff1dc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:25.170407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-6197cce2-dffc-4bdb-b321-81e3cff1dc88 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:25.171635 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-6197cce2-dffc-4bdb-b321-81e3cff1dc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:25.206360 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4e7f9-bd66-4838-8e9a-66846e540e37 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:25.206827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4e7f9-bd66-4838-8e9a-66846e540e37 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:25.207314 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4e7f9-bd66-4838-8e9a-66846e540e37 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:25.208821 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 6/11] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 14:33:25.214998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec86698-c7d6-49a1-953c-a963ab088afd None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:25.215425 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec86698-c7d6-49a1-953c-a963ab088afd None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:25.215834 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec86698-c7d6-49a1-953c-a963ab088afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:25.261902 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG passlib.handlers.bcrypt [None req-dec86698-c7d6-49a1-953c-a963ab088afd None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86861) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 20 14:33:25.262331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG passlib.handlers.bcrypt [None req-dec86698-c7d6-49a1-953c-a963ab088afd None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86861) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 20 14:33:25.262860 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 6/12] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:33:25.285212 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6197cce2-dffc-4bdb-b321-81e3cff1dc88 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:25.308206 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-dec86698-c7d6-49a1-953c-a963ab088afd None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:25.309789 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-dec86698-c7d6-49a1-953c-a963ab088afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:25.309934 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:25.390325 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6197cce2-dffc-4bdb-b321-81e3cff1dc88 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:25.411332 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 7/13] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:33:25.423159 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d49f3d-3f47-4091-a8be-a16a6059a43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:25.423830 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d49f3d-3f47-4091-a8be-a16a6059a43d None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:25.424116 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d49f3d-3f47-4091-a8be-a16a6059a43d None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:25.424766 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d49f3d-3f47-4091-a8be-a16a6059a43d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:25.435577 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec86698-c7d6-49a1-953c-a963ab088afd None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:25.497564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d49f3d-3f47-4091-a8be-a16a6059a43d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:25.499094 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d49f3d-3f47-4091-a8be-a16a6059a43d None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:25.522462 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:25.538614 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec86698-c7d6-49a1-953c-a963ab088afd None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:25.540908 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:25.575982 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a476bd-35ed-4f13-bc42-05e81545940d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:25.577044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a476bd-35ed-4f13-bc42-05e81545940d None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:25.577330 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a476bd-35ed-4f13-bc42-05e81545940d None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:25.577605 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a476bd-35ed-4f13-bc42-05e81545940d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:25.589757 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a476bd-35ed-4f13-bc42-05e81545940d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:33:25.601222 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-48a476bd-35ed-4f13-bc42-05e81545940d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:33:25.601222 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:33:25.601222 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:33:25.601222 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:33:25.601222 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:33:25.601222 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:33:25.601222 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:33:25.601222 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:33:25.601222 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:33:25.601222 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:33:25.601222 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:33:25.601222 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:33:25.601222 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:33:25.601222 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:33:25.601222 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:33:25.601222 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:33:25.602264 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:33:25.602264 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:33:25.602264 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:33:25.602264 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:33:25.602264 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:33:25.602264 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:33:25.602264 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:33:25.602264 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:33:25.602264 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:33:25.602264 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:33:25.602264 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:33:25.602264 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:33:25.602264 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:33:25.602264 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:33:25.602264 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:33:25.602264 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:33:25.603306 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:33:25.603306 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:33:25.603306 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:33:25.603306 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:33:25.603306 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:33:25.603306 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:33:25.603306 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:33:25.603306 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:33:25.603306 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:33:25.603306 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:33:25.603306 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:33:25.603306 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:33:25.603306 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 8/14] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:33:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:33:25.606393 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 7/15] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:33:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 14:33:25.608492 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a476bd-35ed-4f13-bc42-05e81545940d None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:25.612341 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d49f3d-3f47-4091-a8be-a16a6059a43d None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:25.711295 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a476bd-35ed-4f13-bc42-05e81545940d None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:25.722798 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d49f3d-3f47-4091-a8be-a16a6059a43d None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:25.745263 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:25.745539 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9bc5f-617c-4855-971f-fe4ad5d18763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:25.746224 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9bc5f-617c-4855-971f-fe4ad5d18763 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:25.746537 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9bc5f-617c-4855-971f-fe4ad5d18763 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:25.746877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9bc5f-617c-4855-971f-fe4ad5d18763 None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:25.747712 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b9bc5f-617c-4855-971f-fe4ad5d18763 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:25.748016 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b9bc5f-617c-4855-971f-fe4ad5d18763 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:25.749492 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b9bc5f-617c-4855-971f-fe4ad5d18763 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:25.757685 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c27dfb-ff8f-4832-966b-b93293e755bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:25.758502 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c27dfb-ff8f-4832-966b-b93293e755bb None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:25.758785 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c27dfb-ff8f-4832-966b-b93293e755bb None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:25.759083 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c27dfb-ff8f-4832-966b-b93293e755bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:25.761456 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 9/16] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:33:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:25.767171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9bc5f-617c-4855-971f-fe4ad5d18763 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:25.769563 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c27dfb-ff8f-4832-966b-b93293e755bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:25.771029 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c27dfb-ff8f-4832-966b-b93293e755bb None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:25.792305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-53c27dfb-ff8f-4832-966b-b93293e755bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:33:25.792305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:33:25.792305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:33:25.792305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:33:25.792305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:33:25.792305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:33:25.792305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:33:25.792305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:33:25.792305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:33:25.792305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:33:25.792305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:33:25.792305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:33:25.792305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:33:25.792305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:33:25.792305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:33:25.793281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:33:25.793281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:33:25.793281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:33:25.793281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:33:25.793281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:33:25.793281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:33:25.793281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:33:25.793281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:33:25.793281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:33:25.793281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:33:25.793281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:33:25.793281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:33:25.793281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:33:25.793281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:33:25.793281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:33:25.793281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:33:25.793281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:33:25.794092 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:33:25.794092 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:33:25.794092 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:33:25.794092 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:33:25.794092 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 8/17] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:33:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:33:25.795583 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:25.799960 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c27dfb-ff8f-4832-966b-b93293e755bb None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:25.871061 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9bc5f-617c-4855-971f-fe4ad5d18763 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:25.901067 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c27dfb-ff8f-4832-966b-b93293e755bb None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:25.904348 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be1241-f1a4-4f03-a13e-0e2898ab64c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:25.905251 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be1241-f1a4-4f03-a13e-0e2898ab64c1 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:25.905538 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be1241-f1a4-4f03-a13e-0e2898ab64c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:25.905819 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be1241-f1a4-4f03-a13e-0e2898ab64c1 None admin] PATH_INFO: `/v3/users/651b2280231048d59d156e569aa33194` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:25.907528 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be1241-f1a4-4f03-a13e-0e2898ab64c1 None admin] RBAC: Authorizing `identity:get_user(user_id=651b2280231048d59d156e569aa33194)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:25.909021 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be1241-f1a4-4f03-a13e-0e2898ab64c1 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:25.910676 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 10/18] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:33:25 2022] GET /identity/v3/users/651b2280231048d59d156e569aa33194 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:25.916391 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be1241-f1a4-4f03-a13e-0e2898ab64c1 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:25.936712 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbcefcf-0431-4d33-ac92-711557801cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:25.937398 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcefcf-0431-4d33-ac92-711557801cc3 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:25.937685 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcefcf-0431-4d33-ac92-711557801cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:25.937968 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcefcf-0431-4d33-ac92-711557801cc3 None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:25.938709 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbcefcf-0431-4d33-ac92-711557801cc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:25.939021 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbcefcf-0431-4d33-ac92-711557801cc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:25.940466 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbcefcf-0431-4d33-ac92-711557801cc3 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:25.957829 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 9/19] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:33:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:33:26.013444 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:26.022071 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be1241-f1a4-4f03-a13e-0e2898ab64c1 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:26.060780 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69802d3-8c8b-427b-b33c-47d5fe3cf5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:26.061609 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69802d3-8c8b-427b-b33c-47d5fe3cf5aa None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:26.061898 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69802d3-8c8b-427b-b33c-47d5fe3cf5aa None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:26.062175 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69802d3-8c8b-427b-b33c-47d5fe3cf5aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:26.063526 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69802d3-8c8b-427b-b33c-47d5fe3cf5aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:26.065040 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69802d3-8c8b-427b-b33c-47d5fe3cf5aa None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:26.066704 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 11/20] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:33:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:33:26.073017 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbcefcf-0431-4d33-ac92-711557801cc3 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:26.113215 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:26.182984 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbcefcf-0431-4d33-ac92-711557801cc3 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:26.219104 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031563a-276f-43ab-a3a7-6476a0a99278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:26.219836 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031563a-276f-43ab-a3a7-6476a0a99278 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:26.220077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031563a-276f-43ab-a3a7-6476a0a99278 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:26.220358 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031563a-276f-43ab-a3a7-6476a0a99278 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:26.221210 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a031563a-276f-43ab-a3a7-6476a0a99278 None admin] RBAC: Adding query filter params (role.id=49bd5abf255048a0abaa9538e309f516, scope.domain.id=default, user.id=651b2280231048d59d156e569aa33194) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:26.221525 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a031563a-276f-43ab-a3a7-6476a0a99278 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:26.222992 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a031563a-276f-43ab-a3a7-6476a0a99278 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:26.230164 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 10/21] 10.222.0.15 () {56 vars in 1322 bytes} [Wed Jul 20 14:33:26 2022] GET /identity/v3/role_assignments?user.id=651b2280231048d59d156e569aa33194&scope.domain.id=default&role.id=49bd5abf255048a0abaa9538e309f516 => generated 233 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:28.507491 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098b73f-7326-4c30-ac9b-974e9e34d3ae None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:28.508009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098b73f-7326-4c30-ac9b-974e9e34d3ae None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:28.508111 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098b73f-7326-4c30-ac9b-974e9e34d3ae None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:28.509414 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 12/22] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:28.517715 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91844a3-d60a-4534-b86d-5657e49b1efd None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:28.518214 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91844a3-d60a-4534-b86d-5657e49b1efd None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:28.518297 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91844a3-d60a-4534-b86d-5657e49b1efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:28.523598 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666dc53-6a00-4e29-9e96-be449ef6145e None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:28.524108 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666dc53-6a00-4e29-9e96-be449ef6145e None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:28.524592 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666dc53-6a00-4e29-9e96-be449ef6145e None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:28.526196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 13/23] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:28.531559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07972d1-53a2-41c0-b7ae-827d035bdd95 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:28.532122 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07972d1-53a2-41c0-b7ae-827d035bdd95 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:28.532598 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07972d1-53a2-41c0-b7ae-827d035bdd95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:28.548054 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-c91844a3-d60a-4534-b86d-5657e49b1efd None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:28.549634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-c91844a3-d60a-4534-b86d-5657e49b1efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:28.565017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-b07972d1-53a2-41c0-b7ae-827d035bdd95 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:28.566297 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-b07972d1-53a2-41c0-b7ae-827d035bdd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:28.683209 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 11/24] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:33:28.684409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963501c-dd73-487f-b96d-07701e467283 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:28.684728 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963501c-dd73-487f-b96d-07701e467283 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:28.684979 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963501c-dd73-487f-b96d-07701e467283 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:28.686422 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 12/25] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:28.687577 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 14/26] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 20 14:33:28.692805 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e26ed7-3d6f-49ce-88c3-1a8482a869f3 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:28.693283 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e26ed7-3d6f-49ce-88c3-1a8482a869f3 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:28.693712 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e26ed7-3d6f-49ce-88c3-1a8482a869f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:28.711180 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-a7e26ed7-3d6f-49ce-88c3-1a8482a869f3 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:28.712912 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-a7e26ed7-3d6f-49ce-88c3-1a8482a869f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:28.739178 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07972d1-53a2-41c0-b7ae-827d035bdd95 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:28.751318 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:28.826392 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 13/27] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:33:28.826392 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e26ed7-3d6f-49ce-88c3-1a8482a869f3 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:28.886824 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:28.899983 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07972d1-53a2-41c0-b7ae-827d035bdd95 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:28.938716 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fed3ca-b88e-4be6-af9e-28a2b08ced9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:28.939891 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fed3ca-b88e-4be6-af9e-28a2b08ced9b None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:28.940369 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fed3ca-b88e-4be6-af9e-28a2b08ced9b None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:28.940851 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fed3ca-b88e-4be6-af9e-28a2b08ced9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:28.949532 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e26ed7-3d6f-49ce-88c3-1a8482a869f3 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:28.960265 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fed3ca-b88e-4be6-af9e-28a2b08ced9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:28.962056 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fed3ca-b88e-4be6-af9e-28a2b08ced9b None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:28.980708 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-d2fed3ca-b88e-4be6-af9e-28a2b08ced9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:33:28.980708 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:33:28.980708 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:33:28.980708 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:33:28.980708 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:33:28.980708 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:33:28.980708 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:33:28.980708 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:33:28.980708 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:33:28.980708 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:33:28.980708 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:33:28.980708 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:33:28.980708 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:33:28.980708 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:33:28.980708 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:33:28.981556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:33:28.981556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:33:28.981556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:33:28.981556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:33:28.981556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:33:28.981556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:33:28.981556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:33:28.981556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:33:28.981556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:33:28.981556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:33:28.981556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:33:28.981556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:33:28.981556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:33:28.981556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:33:28.981556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:33:28.981556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:33:28.981556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:33:28.982346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:33:28.982346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:33:28.982346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:33:28.982346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:33:28.984297 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 15/28] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:33:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:33:28.985943 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024ee7a-8c29-467d-911f-36b7f35d6d7c None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:28.986044 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4b498-6d7d-4b3b-a062-d3426e566c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:28.986568 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024ee7a-8c29-467d-911f-36b7f35d6d7c None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:28.986718 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4b498-6d7d-4b3b-a062-d3426e566c9e None admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:28.987144 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024ee7a-8c29-467d-911f-36b7f35d6d7c None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:28.987256 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4b498-6d7d-4b3b-a062-d3426e566c9e None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:28.987500 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4b498-6d7d-4b3b-a062-d3426e566c9e None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:28.988261 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b4b498-6d7d-4b3b-a062-d3426e566c9e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:28.989799 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 16/29] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:28.989908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b4b498-6d7d-4b3b-a062-d3426e566c9e None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:28.990793 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024ee7a-8c29-467d-911f-36b7f35d6d7c None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:29.100174 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024ee7a-8c29-467d-911f-36b7f35d6d7c None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:29.125050 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 14/30] 10.222.0.15 () {60 vars in 1131 bytes} [Wed Jul 20 14:33:28 2022] POST /identity/v3/roles => generated 219 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:33:29.126486 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4b498-6d7d-4b3b-a062-d3426e566c9e None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:29.136122 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d14c3-780f-41b3-80fc-5773f6247a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:29.137226 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d14c3-780f-41b3-80fc-5773f6247a56 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:29.137702 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d14c3-780f-41b3-80fc-5773f6247a56 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:29.138152 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d14c3-780f-41b3-80fc-5773f6247a56 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:29.149885 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:29.164548 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d14c3-780f-41b3-80fc-5773f6247a56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:33:29.180819 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-785d14c3-780f-41b3-80fc-5773f6247a56 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:33:29.180819 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:33:29.180819 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:33:29.180819 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:33:29.180819 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:33:29.180819 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:33:29.180819 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:33:29.180819 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:33:29.180819 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:33:29.180819 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:33:29.180819 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:33:29.180819 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:33:29.180819 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:33:29.180819 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:33:29.180819 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:33:29.180819 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:33:29.181809 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:33:29.181809 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:33:29.181809 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:33:29.181809 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:33:29.181809 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:33:29.181809 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:33:29.181809 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:33:29.181809 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:33:29.181809 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:33:29.181809 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:33:29.181809 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:33:29.181809 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:33:29.181809 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:33:29.181809 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:33:29.181809 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:33:29.181809 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:33:29.182551 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:33:29.182551 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:33:29.182551 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:33:29.182551 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:33:29.182551 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:33:29.182551 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:33:29.182551 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:33:29.182551 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:33:29.182551 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:33:29.182551 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:33:29.182551 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:33:29.182551 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:33:29.183587 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 17/31] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:33:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:33:29.183874 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f662cd-5c12-45dc-bfdb-84569ce47a63 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:29.184158 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f662cd-5c12-45dc-bfdb-84569ce47a63 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:29.184435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f662cd-5c12-45dc-bfdb-84569ce47a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:29.200220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:29.220335 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-70f662cd-5c12-45dc-bfdb-84569ce47a63 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:29.221681 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-70f662cd-5c12-45dc-bfdb-84569ce47a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:29.248705 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4b498-6d7d-4b3b-a062-d3426e566c9e None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:29.285490 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551a646-2dd1-4c5c-b99a-a4c6280f3f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:29.286475 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551a646-2dd1-4c5c-b99a-a4c6280f3f22 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:29.287095 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551a646-2dd1-4c5c-b99a-a4c6280f3f22 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:29.287549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551a646-2dd1-4c5c-b99a-a4c6280f3f22 None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:29.288599 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551a646-2dd1-4c5c-b99a-a4c6280f3f22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:29.289093 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551a646-2dd1-4c5c-b99a-a4c6280f3f22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:29.290816 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551a646-2dd1-4c5c-b99a-a4c6280f3f22 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:29.306004 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 15/32] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:29.307703 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551a646-2dd1-4c5c-b99a-a4c6280f3f22 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:29.378544 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 18/33] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:33:29.389006 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f662cd-5c12-45dc-bfdb-84569ce47a63 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:29.428089 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:29.443407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551a646-2dd1-4c5c-b99a-a4c6280f3f22 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:29.488223 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a76219-425e-4839-bb69-37e8e36292c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:29.488751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a76219-425e-4839-bb69-37e8e36292c4 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:29.489041 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a76219-425e-4839-bb69-37e8e36292c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:29.489324 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a76219-425e-4839-bb69-37e8e36292c4 None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:29.491296 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a76219-425e-4839-bb69-37e8e36292c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:29.491620 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a76219-425e-4839-bb69-37e8e36292c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:29.493235 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a76219-425e-4839-bb69-37e8e36292c4 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:29.513826 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 16/34] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:33:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:29.513826 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a76219-425e-4839-bb69-37e8e36292c4 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:29.546111 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f662cd-5c12-45dc-bfdb-84569ce47a63 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:29.582609 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b5d76-6568-44b3-8c78-c8d4afd3523f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:29.583698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b5d76-6568-44b3-8c78-c8d4afd3523f None admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:29.584193 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b5d76-6568-44b3-8c78-c8d4afd3523f None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:29.584675 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b5d76-6568-44b3-8c78-c8d4afd3523f None admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:29.585698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50b5d76-6568-44b3-8c78-c8d4afd3523f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:29.587374 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50b5d76-6568-44b3-8c78-c8d4afd3523f None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:29.591650 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:29.638741 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a76219-425e-4839-bb69-37e8e36292c4 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:29.676050 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf82fb4-5cbc-4bfc-8d71-6de99f61db6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:29.677108 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf82fb4-5cbc-4bfc-8d71-6de99f61db6a None admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:29.677624 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf82fb4-5cbc-4bfc-8d71-6de99f61db6a None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:29.678108 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf82fb4-5cbc-4bfc-8d71-6de99f61db6a None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:29.679171 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf82fb4-5cbc-4bfc-8d71-6de99f61db6a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:29.680885 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf82fb4-5cbc-4bfc-8d71-6de99f61db6a None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:29.693101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:29.699019 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 17/35] 10.222.0.15 () {60 vars in 1131 bytes} [Wed Jul 20 14:33:29 2022] POST /identity/v3/roles => generated 225 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:33:29.700468 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf82fb4-5cbc-4bfc-8d71-6de99f61db6a None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:29.758830 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 19/36] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Jul 20 14:33:29 2022] POST /identity/v3/projects => generated 301 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:33:29.896622 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6cbe8-4615-4561-b118-caf6b6fbcf1f None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:29.897063 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6cbe8-4615-4561-b118-caf6b6fbcf1f None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:29.897284 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6cbe8-4615-4561-b118-caf6b6fbcf1f None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:29.898770 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 20/37] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 14:33:29.905178 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06296816-64ce-43fe-b53e-a5cd6948ffc3 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:29.905618 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06296816-64ce-43fe-b53e-a5cd6948ffc3 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:29.906008 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06296816-64ce-43fe-b53e-a5cd6948ffc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:29.926133 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-06296816-64ce-43fe-b53e-a5cd6948ffc3 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:29.928058 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-06296816-64ce-43fe-b53e-a5cd6948ffc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:29.952812 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf82fb4-5cbc-4bfc-8d71-6de99f61db6a None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:30.002453 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f37653-4427-45c7-a7b0-0224e51dc4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:30.004087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f37653-4427-45c7-a7b0-0224e51dc4b0 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:30.004393 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f37653-4427-45c7-a7b0-0224e51dc4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:30.004663 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f37653-4427-45c7-a7b0-0224e51dc4b0 None admin] PATH_INFO: `/v3/users/651b2280231048d59d156e569aa33194` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:30.007876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f37653-4427-45c7-a7b0-0224e51dc4b0 None admin] RBAC: Authorizing `identity:get_user(user_id=651b2280231048d59d156e569aa33194)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:30.007876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f37653-4427-45c7-a7b0-0224e51dc4b0 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:30.009341 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 18/38] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:33:29 2022] GET /identity/v3/users/651b2280231048d59d156e569aa33194 => generated 247 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:30.014919 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f37653-4427-45c7-a7b0-0224e51dc4b0 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:30.034904 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 21/39] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 14:33:30.067111 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06296816-64ce-43fe-b53e-a5cd6948ffc3 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:30.093030 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:30.129997 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f37653-4427-45c7-a7b0-0224e51dc4b0 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:30.163694 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821a6d4-5c8e-416a-9c9c-c2a60c493d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:30.164503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821a6d4-5c8e-416a-9c9c-c2a60c493d07 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:30.164800 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821a6d4-5c8e-416a-9c9c-c2a60c493d07 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:30.165085 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821a6d4-5c8e-416a-9c9c-c2a60c493d07 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:30.166381 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0821a6d4-5c8e-416a-9c9c-c2a60c493d07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:30.167902 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0821a6d4-5c8e-416a-9c9c-c2a60c493d07 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:30.169563 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 19/40] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:33:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:30.170926 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda3991-8b81-4f06-82ec-9a77f2eb62a6 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:30.171249 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda3991-8b81-4f06-82ec-9a77f2eb62a6 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:30.171570 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda3991-8b81-4f06-82ec-9a77f2eb62a6 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:30.172785 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 20/41] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:30.175653 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda3991-8b81-4f06-82ec-9a77f2eb62a6 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:30.180359 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06296816-64ce-43fe-b53e-a5cd6948ffc3 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:30.199425 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:30.219209 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328369ef-8cb3-4132-a989-a78b620334fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:30.219899 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328369ef-8cb3-4132-a989-a78b620334fa None admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:30.220195 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328369ef-8cb3-4132-a989-a78b620334fa None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:30.220550 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328369ef-8cb3-4132-a989-a78b620334fa None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:30.221314 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328369ef-8cb3-4132-a989-a78b620334fa None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:30.222857 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328369ef-8cb3-4132-a989-a78b620334fa None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:30.237993 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 22/42] 10.222.0.15 () {60 vars in 1131 bytes} [Wed Jul 20 14:33:30 2022] POST /identity/v3/roles => generated 223 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 14:33:30.237993 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b03f3-9aa7-49d5-aa65-499eb3bde2bd None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:30.237993 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b03f3-9aa7-49d5-aa65-499eb3bde2bd None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:30.237993 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b03f3-9aa7-49d5-aa65-499eb3bde2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:30.268153 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-e70b03f3-9aa7-49d5-aa65-499eb3bde2bd None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:30.269808 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-e70b03f3-9aa7-49d5-aa65-499eb3bde2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:30.292344 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda3991-8b81-4f06-82ec-9a77f2eb62a6 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:30.332344 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07847bd-b791-49bd-9092-0e6993cf4134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:30.333278 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07847bd-b791-49bd-9092-0e6993cf4134 None admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:30.333557 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07847bd-b791-49bd-9092-0e6993cf4134 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:30.333894 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07847bd-b791-49bd-9092-0e6993cf4134 None admin] PATH_INFO: `/v3/domains/default/users/651b2280231048d59d156e569aa33194/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:30.336252 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07847bd-b791-49bd-9092-0e6993cf4134 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=651b2280231048d59d156e569aa33194, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:30.337739 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07847bd-b791-49bd-9092-0e6993cf4134 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:30.342739 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:30.357977 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 21/43] 10.222.0.15 () {58 vars in 1361 bytes} [Wed Jul 20 14:33:30 2022] PUT /identity/v3/domains/default/users/651b2280231048d59d156e569aa33194/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 14:33:30.368885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 23/44] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:33:30.392355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07847bd-b791-49bd-9092-0e6993cf4134 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:30.506974 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07847bd-b791-49bd-9092-0e6993cf4134 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:30.542000 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5fdf5-7d70-4a34-a4a1-29be2013ae51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:30.542851 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5fdf5-7d70-4a34-a4a1-29be2013ae51 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:30.543166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5fdf5-7d70-4a34-a4a1-29be2013ae51 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:30.543446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5fdf5-7d70-4a34-a4a1-29be2013ae51 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:30.544640 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b5fdf5-7d70-4a34-a4a1-29be2013ae51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:30.546118 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b5fdf5-7d70-4a34-a4a1-29be2013ae51 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:30.547790 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 22/45] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:33:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:30.553910 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b03f3-9aa7-49d5-aa65-499eb3bde2bd None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:30.656588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b03f3-9aa7-49d5-aa65-499eb3bde2bd None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:30.664401 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:30.692519 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31b568-8bd3-4412-bc6a-956296d973be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:30.693198 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31b568-8bd3-4412-bc6a-956296d973be None admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:30.693487 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31b568-8bd3-4412-bc6a-956296d973be None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:30.693767 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31b568-8bd3-4412-bc6a-956296d973be None admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:30.694550 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e31b568-8bd3-4412-bc6a-956296d973be None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:30.696064 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e31b568-8bd3-4412-bc6a-956296d973be None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:30.722690 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 24/46] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Jul 20 14:33:30 2022] POST /identity/v3/projects => generated 312 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:33:32.394618 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d1e12-7914-4fa5-aac7-43fa38ee83a7 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:32.395187 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d1e12-7914-4fa5-aac7-43fa38ee83a7 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:32.395187 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d1e12-7914-4fa5-aac7-43fa38ee83a7 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:32.396797 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 23/47] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:32.402202 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db0da6-a657-442c-bcc3-387b4deaf077 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:32.402655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db0da6-a657-442c-bcc3-387b4deaf077 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:32.402804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db0da6-a657-442c-bcc3-387b4deaf077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:32.420887 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-b7db0da6-a657-442c-bcc3-387b4deaf077 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:32.422093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-b7db0da6-a657-442c-bcc3-387b4deaf077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:32.496927 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 25/48] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:33:32.520284 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d1e12-7914-4fa5-aac7-43fa38ee83a7 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:32.586682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:32.630473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d1e12-7914-4fa5-aac7-43fa38ee83a7 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:32.666217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c60675-caaa-4ff7-aafd-8133748413e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:32.667029 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c60675-caaa-4ff7-aafd-8133748413e2 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:32.667382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c60675-caaa-4ff7-aafd-8133748413e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:32.667664 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c60675-caaa-4ff7-aafd-8133748413e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:32.680120 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c60675-caaa-4ff7-aafd-8133748413e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:33:32.689144 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-30c60675-caaa-4ff7-aafd-8133748413e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:33:32.689144 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:33:32.689144 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:33:32.689144 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:33:32.689144 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:33:32.689144 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:33:32.689144 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:33:32.689144 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:33:32.689144 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:33:32.689144 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:33:32.689144 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:33:32.689144 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:33:32.689144 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:33:32.689144 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:33:32.689144 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:33:32.689144 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:33:32.690081 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:33:32.690081 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:33:32.690081 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:33:32.690081 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:33:32.690081 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:33:32.690081 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:33:32.690081 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:33:32.690081 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:33:32.690081 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:33:32.690081 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:33:32.690081 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:33:32.690081 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:33:32.690081 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:33:32.690081 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:33:32.690081 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:33:32.690081 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:33:32.690923 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:33:32.690923 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:33:32.690923 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:33:32.690923 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:33:32.690923 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:33:32.690923 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:33:32.690923 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:33:32.690923 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:33:32.690923 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:33:32.690923 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:33:32.690923 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:33:32.690923 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:33:32.690923 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 24/49] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:33:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:33:32.696615 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db0da6-a657-442c-bcc3-387b4deaf077 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:32.720858 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba826f4e-4b16-44b8-b7f9-0df2dd07e542 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:32.721148 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba826f4e-4b16-44b8-b7f9-0df2dd07e542 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:32.721429 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba826f4e-4b16-44b8-b7f9-0df2dd07e542 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:32.723183 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 25/50] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:32.728331 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3de9c5-7f65-43e2-9e6f-97900b73df81 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:32.728614 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3de9c5-7f65-43e2-9e6f-97900b73df81 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:32.728903 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3de9c5-7f65-43e2-9e6f-97900b73df81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:32.747443 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-db3de9c5-7f65-43e2-9e6f-97900b73df81 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:32.749253 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-db3de9c5-7f65-43e2-9e6f-97900b73df81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:32.788396 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:32.801915 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db0da6-a657-442c-bcc3-387b4deaf077 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:32.819118 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 26/51] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 14:33:32.838290 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce533b40-08e3-4111-ab47-883fb77b2fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:32.838971 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce533b40-08e3-4111-ab47-883fb77b2fba None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:32.839283 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce533b40-08e3-4111-ab47-883fb77b2fba None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:32.839565 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce533b40-08e3-4111-ab47-883fb77b2fba None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:32.840519 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce533b40-08e3-4111-ab47-883fb77b2fba None admin] RBAC: Adding query filter params (name=admin) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:32.840621 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce533b40-08e3-4111-ab47-883fb77b2fba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:32.841475 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3de9c5-7f65-43e2-9e6f-97900b73df81 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:32.842009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce533b40-08e3-4111-ab47-883fb77b2fba None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:32.856995 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 26/52] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:33:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:32.858115 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce533b40-08e3-4111-ab47-883fb77b2fba None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:32.895805 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:32.941794 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3de9c5-7f65-43e2-9e6f-97900b73df81 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:32.964173 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce533b40-08e3-4111-ab47-883fb77b2fba None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:32.977599 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc383ba1-a9d1-45fb-9401-77d2376f59f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:32.978419 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc383ba1-a9d1-45fb-9401-77d2376f59f2 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:32.978708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc383ba1-a9d1-45fb-9401-77d2376f59f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:32.978996 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc383ba1-a9d1-45fb-9401-77d2376f59f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:32.980262 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc383ba1-a9d1-45fb-9401-77d2376f59f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:32.981744 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc383ba1-a9d1-45fb-9401-77d2376f59f2 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:32.983617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 27/53] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:33:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:33:32.989038 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc383ba1-a9d1-45fb-9401-77d2376f59f2 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:32.997995 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7288b4b-5cd9-48ff-b661-f139200c1c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:32.998877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7288b4b-5cd9-48ff-b661-f139200c1c11 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:32.999190 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7288b4b-5cd9-48ff-b661-f139200c1c11 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:32.999468 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7288b4b-5cd9-48ff-b661-f139200c1c11 None admin] PATH_INFO: `/v3/users/651b2280231048d59d156e569aa33194` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:33.000677 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7288b4b-5cd9-48ff-b661-f139200c1c11 None admin] RBAC: Authorizing `identity:get_user(user_id=651b2280231048d59d156e569aa33194)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:33.002145 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7288b4b-5cd9-48ff-b661-f139200c1c11 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:33.003830 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 27/54] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:33:32 2022] GET /identity/v3/users/651b2280231048d59d156e569aa33194 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:33.009267 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7288b4b-5cd9-48ff-b661-f139200c1c11 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:33.029132 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:33.094127 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc383ba1-a9d1-45fb-9401-77d2376f59f2 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:33.119625 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:33.132412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7288b4b-5cd9-48ff-b661-f139200c1c11 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:33.136063 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1c1de-f818-4577-b2a8-2e6fbbaed228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:33.136752 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1c1de-f818-4577-b2a8-2e6fbbaed228 None admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:33.137039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1c1de-f818-4577-b2a8-2e6fbbaed228 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:33.137323 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1c1de-f818-4577-b2a8-2e6fbbaed228 None admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:33.138058 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa1c1de-f818-4577-b2a8-2e6fbbaed228 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:33.139559 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa1c1de-f818-4577-b2a8-2e6fbbaed228 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:33.171652 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 28/55] 10.222.0.15 () {60 vars in 1140 bytes} [Wed Jul 20 14:33:32 2022] POST /identity/v3/projects => generated 298 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 14:33:33.178700 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd0610-4dd7-48df-99e7-51460eab5f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:33.179611 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd0610-4dd7-48df-99e7-51460eab5f6a None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:33.179906 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd0610-4dd7-48df-99e7-51460eab5f6a None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:33.180212 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd0610-4dd7-48df-99e7-51460eab5f6a None admin] PATH_INFO: `/v3/domains/default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:33.181571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd0610-4dd7-48df-99e7-51460eab5f6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:33.183145 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd0610-4dd7-48df-99e7-51460eab5f6a None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:33.185246 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 28/56] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:33:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:33.192047 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1c1de-f818-4577-b2a8-2e6fbbaed228 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:33.314106 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1c1de-f818-4577-b2a8-2e6fbbaed228 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:33.350018 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596bd9bf-3abe-488f-b2c3-e98993e1868e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:33.350808 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596bd9bf-3abe-488f-b2c3-e98993e1868e None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:33.351170 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596bd9bf-3abe-488f-b2c3-e98993e1868e None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:33.351506 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596bd9bf-3abe-488f-b2c3-e98993e1868e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:33.352446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596bd9bf-3abe-488f-b2c3-e98993e1868e None admin] RBAC: Adding query filter params (role.id=49bd5abf255048a0abaa9538e309f516, scope.domain.id=default, user.id=651b2280231048d59d156e569aa33194) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:33.352749 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596bd9bf-3abe-488f-b2c3-e98993e1868e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:33.354329 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596bd9bf-3abe-488f-b2c3-e98993e1868e None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:33.360459 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 29/57] 10.222.0.15 () {56 vars in 1322 bytes} [Wed Jul 20 14:33:33 2022] GET /identity/v3/role_assignments?user.id=651b2280231048d59d156e569aa33194&scope.domain.id=default&role.id=49bd5abf255048a0abaa9538e309f516 => generated 530 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:35.256605 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9eb13b-b610-45a7-8cee-ee9075082def None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:35.257289 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9eb13b-b610-45a7-8cee-ee9075082def None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:35.257426 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9eb13b-b610-45a7-8cee-ee9075082def None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:35.259728 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 29/58] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:35.265970 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616054c-be2d-42cd-93b0-93676f74bca2 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:35.266395 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616054c-be2d-42cd-93b0-93676f74bca2 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:35.266608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616054c-be2d-42cd-93b0-93676f74bca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:35.288156 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-e616054c-be2d-42cd-93b0-93676f74bca2 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:35.289383 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e616054c-be2d-42cd-93b0-93676f74bca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:35.322711 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:35.364901 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 30/59] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:33:35.389841 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9eb13b-b610-45a7-8cee-ee9075082def None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:35.492066 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:35.510237 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9eb13b-b610-45a7-8cee-ee9075082def None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:35.545202 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2bebcd-eb21-4fa4-9291-62fcb9cca128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:35.546001 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bebcd-eb21-4fa4-9291-62fcb9cca128 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:35.546292 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bebcd-eb21-4fa4-9291-62fcb9cca128 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:35.546566 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bebcd-eb21-4fa4-9291-62fcb9cca128 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:35.548515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2bebcd-eb21-4fa4-9291-62fcb9cca128 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:35.551439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2bebcd-eb21-4fa4-9291-62fcb9cca128 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:35.553400 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 30/60] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:33:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:35.561926 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616054c-be2d-42cd-93b0-93676f74bca2 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:35.677779 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616054c-be2d-42cd-93b0-93676f74bca2 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:35.718296 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c497387-0e39-4381-9064-627f7feb529d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:35.719359 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c497387-0e39-4381-9064-627f7feb529d None admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:35.719755 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c497387-0e39-4381-9064-627f7feb529d None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:35.720121 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c497387-0e39-4381-9064-627f7feb529d None admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:35.721242 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c497387-0e39-4381-9064-627f7feb529d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:35.723671 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c497387-0e39-4381-9064-627f7feb529d None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:35.757194 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 31/61] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jul 20 14:33:35 2022] POST /identity/v3/users => generated 275 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:33:38.946468 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485722c-48cf-4963-9061-42621932f6f6 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:38.947537 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485722c-48cf-4963-9061-42621932f6f6 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:38.948016 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485722c-48cf-4963-9061-42621932f6f6 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:38.949597 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 31/62] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:38.955005 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d194b00-3133-4f56-97b7-9248c276ec34 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:38.955475 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d194b00-3133-4f56-97b7-9248c276ec34 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:38.955579 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d194b00-3133-4f56-97b7-9248c276ec34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:38.980091 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-2d194b00-3133-4f56-97b7-9248c276ec34 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:38.981303 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-2d194b00-3133-4f56-97b7-9248c276ec34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:39.006132 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:39.034669 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b0c5e-b3a7-49d6-a0a3-86e224b98c5d None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:39.034970 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b0c5e-b3a7-49d6-a0a3-86e224b98c5d None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:39.035257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b0c5e-b3a7-49d6-a0a3-86e224b98c5d None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:39.036632 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 32/63] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 14:33:39.042208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc82742-8a9f-442d-9b4e-f01289e607c1 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:39.042832 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc82742-8a9f-442d-9b4e-f01289e607c1 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:39.043307 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc82742-8a9f-442d-9b4e-f01289e607c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:39.088684 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 32/64] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:33:39.136990 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d194b00-3133-4f56-97b7-9248c276ec34 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:39.227245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-5fc82742-8a9f-442d-9b4e-f01289e607c1 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:39.228408 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-5fc82742-8a9f-442d-9b4e-f01289e607c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:39.298956 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d194b00-3133-4f56-97b7-9248c276ec34 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:39.332319 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 33/65] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:33:39.334109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a1ea0e-5d6d-41f0-9c55-0f1582d7ef3d None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:39.334563 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a1ea0e-5d6d-41f0-9c55-0f1582d7ef3d None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:39.335040 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a1ea0e-5d6d-41f0-9c55-0f1582d7ef3d None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:39.336846 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be228874-47b9-453b-87b7-20a9357204ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:39.337155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 34/66] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:39.337989 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be228874-47b9-453b-87b7-20a9357204ae None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:39.338476 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be228874-47b9-453b-87b7-20a9357204ae None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:39.338943 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be228874-47b9-453b-87b7-20a9357204ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:39.342824 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbd787-40f6-4783-84d4-3e9e33a5386f None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:39.343323 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbd787-40f6-4783-84d4-3e9e33a5386f None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:39.343844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbd787-40f6-4783-84d4-3e9e33a5386f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:39.353313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be228874-47b9-453b-87b7-20a9357204ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:33:39.361062 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-5bdbd787-40f6-4783-84d4-3e9e33a5386f None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:39.362265 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-5bdbd787-40f6-4783-84d4-3e9e33a5386f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:39.362794 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-be228874-47b9-453b-87b7-20a9357204ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:33:39.362794 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:33:39.362794 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:33:39.362794 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:33:39.362794 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:33:39.362794 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:33:39.362794 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:33:39.362794 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:33:39.362794 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:33:39.362794 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:33:39.362794 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:33:39.362794 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:33:39.362794 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:33:39.362794 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:33:39.362794 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:33:39.362794 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:33:39.363637 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:33:39.363637 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:33:39.363637 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:33:39.363637 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:33:39.363637 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:33:39.363637 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:33:39.363637 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:33:39.363637 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:33:39.363637 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:33:39.363637 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:33:39.363637 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:33:39.363637 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:33:39.363637 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:33:39.363637 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:33:39.363637 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:33:39.363637 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:33:39.364576 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:33:39.364576 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:33:39.364576 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:33:39.364576 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:33:39.364576 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:33:39.364576 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:33:39.364576 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:33:39.364576 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:33:39.364576 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:33:39.364576 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:33:39.364576 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:33:39.364576 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:33:39.368086 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 33/67] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:33:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:33:39.372723 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be228874-47b9-453b-87b7-20a9357204ae None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:39.394677 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:39.439102 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 35/68] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 14:33:39.448127 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdbd787-40f6-4783-84d4-3e9e33a5386f None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:39.475658 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:39.508245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be228874-47b9-453b-87b7-20a9357204ae None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:39.543152 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68417dc-b07a-468f-a076-4e20fc007b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:39.543965 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68417dc-b07a-468f-a076-4e20fc007b19 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:39.544250 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68417dc-b07a-468f-a076-4e20fc007b19 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:39.544529 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68417dc-b07a-468f-a076-4e20fc007b19 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:39.555234 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68417dc-b07a-468f-a076-4e20fc007b19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:33:39.562823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-d68417dc-b07a-468f-a076-4e20fc007b19 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:33:39.562823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:33:39.562823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:33:39.562823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:33:39.562823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:33:39.562823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:33:39.562823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:33:39.562823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:33:39.562823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:33:39.562823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:33:39.562823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:33:39.562823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:33:39.562823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:33:39.562823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:33:39.562823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:33:39.562823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:33:39.563725 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:33:39.563725 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:33:39.563725 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:33:39.563725 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:33:39.563725 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:33:39.563725 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:33:39.563725 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:33:39.563725 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:33:39.563725 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:33:39.563725 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:33:39.563725 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:33:39.563725 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:33:39.563725 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:33:39.563725 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:33:39.563725 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:33:39.563725 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:33:39.564790 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:33:39.564790 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:33:39.564790 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:33:39.564790 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:33:39.564790 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:33:39.564790 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:33:39.564790 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:33:39.564790 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:33:39.564790 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:33:39.564790 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:33:39.564790 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:33:39.564790 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:33:39.564790 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 34/69] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:33:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:33:39.565566 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68417dc-b07a-468f-a076-4e20fc007b19 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:39.573352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdbd787-40f6-4783-84d4-3e9e33a5386f None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:39.610417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feebd858-1542-4955-81d2-1a3f958d74a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:39.611135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feebd858-1542-4955-81d2-1a3f958d74a1 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:39.611423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feebd858-1542-4955-81d2-1a3f958d74a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:39.611774 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feebd858-1542-4955-81d2-1a3f958d74a1 None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:39.612451 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feebd858-1542-4955-81d2-1a3f958d74a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:39.612759 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feebd858-1542-4955-81d2-1a3f958d74a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:39.614158 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feebd858-1542-4955-81d2-1a3f958d74a1 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:39.623896 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 36/70] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:33:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:39.625000 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386ff83-a11b-4798-aec6-03db8165726e None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:39.625290 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386ff83-a11b-4798-aec6-03db8165726e None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:39.625568 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386ff83-a11b-4798-aec6-03db8165726e None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:39.626932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 37/71] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:39.631426 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386ff83-a11b-4798-aec6-03db8165726e None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:39.648538 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:39.677706 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68417dc-b07a-468f-a076-4e20fc007b19 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:39.714288 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9412604-e0af-4d6f-b836-3bd5617ef163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:39.715160 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9412604-e0af-4d6f-b836-3bd5617ef163 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:39.715160 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9412604-e0af-4d6f-b836-3bd5617ef163 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:39.715455 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9412604-e0af-4d6f-b836-3bd5617ef163 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:39.717794 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:39.728836 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9412604-e0af-4d6f-b836-3bd5617ef163 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:33:39.736881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-b9412604-e0af-4d6f-b836-3bd5617ef163 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 14:33:39.736881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:33:39.736881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:33:39.736881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:33:39.736881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:33:39.736881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:33:39.736881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:33:39.736881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:33:39.736881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:33:39.736881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:33:39.736881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:33:39.736881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:33:39.736881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:33:39.736881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:33:39.736881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:33:39.736881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:33:39.737741 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:33:39.737741 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:33:39.737741 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:33:39.737741 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:33:39.737741 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:33:39.737741 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:33:39.737741 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:33:39.737741 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:33:39.737741 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:33:39.737741 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:33:39.737741 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:33:39.737741 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:33:39.737741 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:33:39.737741 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:33:39.737741 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:33:39.737741 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:33:39.738529 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:33:39.738529 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:33:39.738529 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:33:39.738529 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:33:39.738529 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:33:39.738529 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:33:39.738529 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:33:39.738529 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:33:39.738529 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:33:39.738529 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:33:39.738529 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 14:33:39.738529 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:33:39.738529 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 35/72] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jul 20 14:33:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:33:39.739847 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9412604-e0af-4d6f-b836-3bd5617ef163 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:39.750657 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386ff83-a11b-4798-aec6-03db8165726e None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:39.786167 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94527dd-a11f-4657-9ff1-e94fffe4d03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:39.786862 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94527dd-a11f-4657-9ff1-e94fffe4d03c None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:39.787187 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94527dd-a11f-4657-9ff1-e94fffe4d03c None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:39.787472 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94527dd-a11f-4657-9ff1-e94fffe4d03c None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:39.788240 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94527dd-a11f-4657-9ff1-e94fffe4d03c None admin] RBAC: Adding query filter params (name=member) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:39.788551 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94527dd-a11f-4657-9ff1-e94fffe4d03c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:39.790002 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94527dd-a11f-4657-9ff1-e94fffe4d03c None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:39.802332 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 38/73] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jul 20 14:33:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:39.803632 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380ecc6-7f22-4ff8-bd93-d85041b6577e None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:39.803924 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380ecc6-7f22-4ff8-bd93-d85041b6577e None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:39.804206 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380ecc6-7f22-4ff8-bd93-d85041b6577e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:39.825482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-5380ecc6-7f22-4ff8-bd93-d85041b6577e None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:39.826840 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-5380ecc6-7f22-4ff8-bd93-d85041b6577e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:39.846518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9412604-e0af-4d6f-b836-3bd5617ef163 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:39.882109 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b2457c-4a6e-483c-b677-f3a3bb9b55ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:39.883007 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2457c-4a6e-483c-b677-f3a3bb9b55ae None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:39.883323 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2457c-4a6e-483c-b677-f3a3bb9b55ae None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:39.883604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2457c-4a6e-483c-b677-f3a3bb9b55ae None admin] PATH_INFO: `/v3/users/651b2280231048d59d156e569aa33194` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:39.884841 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b2457c-4a6e-483c-b677-f3a3bb9b55ae None admin] RBAC: Authorizing `identity:get_user(user_id=651b2280231048d59d156e569aa33194)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:39.886323 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b2457c-4a6e-483c-b677-f3a3bb9b55ae None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:39.892427 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 36/74] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:33:39 2022] GET /identity/v3/users/651b2280231048d59d156e569aa33194 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:39.892427 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b2457c-4a6e-483c-b677-f3a3bb9b55ae None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:39.892427 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:39.894875 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 39/75] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:33:39.896260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb288f-ecf7-4c83-919d-97e1ea17d8e9 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:39.896556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb288f-ecf7-4c83-919d-97e1ea17d8e9 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:39.896837 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb288f-ecf7-4c83-919d-97e1ea17d8e9 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:39.898234 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 40/76] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:39.899840 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb288f-ecf7-4c83-919d-97e1ea17d8e9 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:39.969140 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:39.997090 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b2457c-4a6e-483c-b677-f3a3bb9b55ae None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:40.002572 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb288f-ecf7-4c83-919d-97e1ea17d8e9 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:40.031569 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe9074-31c2-4d43-acb7-dacd3a4332b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:40.032253 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe9074-31c2-4d43-acb7-dacd3a4332b2 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:40.032540 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe9074-31c2-4d43-acb7-dacd3a4332b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:40.032817 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe9074-31c2-4d43-acb7-dacd3a4332b2 None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:40.033581 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe9074-31c2-4d43-acb7-dacd3a4332b2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:40.033890 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe9074-31c2-4d43-acb7-dacd3a4332b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:40.035325 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe9074-31c2-4d43-acb7-dacd3a4332b2 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:40.038056 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53037dd-f8a3-4980-8997-7e015985b8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:40.038978 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53037dd-f8a3-4980-8997-7e015985b8aa None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:40.039294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53037dd-f8a3-4980-8997-7e015985b8aa None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:40.039572 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53037dd-f8a3-4980-8997-7e015985b8aa None admin] PATH_INFO: `/v3/users/55e7a5b2a35b46a0a3eedafbe6c53a2c` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:40.043100 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 37/77] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:33:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:40.043903 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe9074-31c2-4d43-acb7-dacd3a4332b2 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:40.059361 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53037dd-f8a3-4980-8997-7e015985b8aa None admin] RBAC: Authorizing `identity:get_user(user_id=55e7a5b2a35b46a0a3eedafbe6c53a2c)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:40.060876 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53037dd-f8a3-4980-8997-7e015985b8aa None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:40.062619 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 41/78] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:33:39 2022] GET /identity/v3/users/55e7a5b2a35b46a0a3eedafbe6c53a2c => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:40.063928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d6846-92c4-48d8-9143-900ccbffa626 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:40.064222 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d6846-92c4-48d8-9143-900ccbffa626 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:40.064502 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d6846-92c4-48d8-9143-900ccbffa626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:40.080658 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-208d6846-92c4-48d8-9143-900ccbffa626 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:40.081854 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-208d6846-92c4-48d8-9143-900ccbffa626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:40.110133 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:40.148420 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe9074-31c2-4d43-acb7-dacd3a4332b2 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:40.151175 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 42/79] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:33:40.152053 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208d6846-92c4-48d8-9143-900ccbffa626 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:40.186860 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08cfc4-0091-4d7c-9ca8-f59044996e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:40.187570 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08cfc4-0091-4d7c-9ca8-f59044996e28 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:40.187876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08cfc4-0091-4d7c-9ca8-f59044996e28 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:40.188157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08cfc4-0091-4d7c-9ca8-f59044996e28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:40.209311 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:40.252514 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208d6846-92c4-48d8-9143-900ccbffa626 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:40.256037 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf08cfc4-0091-4d7c-9ca8-f59044996e28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:40.257530 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf08cfc4-0091-4d7c-9ca8-f59044996e28 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:40.286240 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929cfdc-ac55-44a8-9024-6014c0ddcae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:40.287192 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929cfdc-ac55-44a8-9024-6014c0ddcae4 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:40.287476 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929cfdc-ac55-44a8-9024-6014c0ddcae4 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:40.287748 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929cfdc-ac55-44a8-9024-6014c0ddcae4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:40.298089 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929cfdc-ac55-44a8-9024-6014c0ddcae4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:33:40.306258 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-4929cfdc-ac55-44a8-9024-6014c0ddcae4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:33:40.306258 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:33:40.306258 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:33:40.306258 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:33:40.306258 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:33:40.306258 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:33:40.306258 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:33:40.306258 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:33:40.306258 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:33:40.306258 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:33:40.306258 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:33:40.306258 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:33:40.306258 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:33:40.306258 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:33:40.306258 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:33:40.306258 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:33:40.307304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:33:40.307304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:33:40.307304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:33:40.307304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:33:40.307304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:33:40.307304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:33:40.307304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:33:40.307304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:33:40.307304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:33:40.307304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:33:40.307304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:33:40.307304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:33:40.307304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:33:40.307304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:33:40.307304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:33:40.307304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:33:40.313286 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:33:40.313286 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:33:40.313286 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:33:40.313286 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:33:40.313286 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:33:40.313286 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:33:40.313286 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:33:40.313286 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:33:40.313286 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:33:40.313286 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:33:40.313286 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:33:40.313286 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:33:40.313286 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 43/80] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:33:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:33:40.313286 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929cfdc-ac55-44a8-9024-6014c0ddcae4 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:40.318070 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:40.356452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 38/81] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:33:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:33:40.357287 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08cfc4-0091-4d7c-9ca8-f59044996e28 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:40.417460 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929cfdc-ac55-44a8-9024-6014c0ddcae4 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:40.434700 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:40.453456 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b6c46f-23ee-4f35-b95d-e1b1f5075c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:40.454356 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6c46f-23ee-4f35-b95d-e1b1f5075c09 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:40.454665 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6c46f-23ee-4f35-b95d-e1b1f5075c09 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:40.454943 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6c46f-23ee-4f35-b95d-e1b1f5075c09 None admin] PATH_INFO: `/v3/users/55e7a5b2a35b46a0a3eedafbe6c53a2c` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:40.456205 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b6c46f-23ee-4f35-b95d-e1b1f5075c09 None admin] RBAC: Authorizing `identity:get_user(user_id=55e7a5b2a35b46a0a3eedafbe6c53a2c)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:40.457390 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08cfc4-0091-4d7c-9ca8-f59044996e28 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:40.457695 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b6c46f-23ee-4f35-b95d-e1b1f5075c09 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:40.463898 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 44/82] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:33:40 2022] GET /identity/v3/users/55e7a5b2a35b46a0a3eedafbe6c53a2c => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:40.463898 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b6c46f-23ee-4f35-b95d-e1b1f5075c09 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:40.498908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5218903-54ba-45af-a1d1-d5bbc258dbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:40.499622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5218903-54ba-45af-a1d1-d5bbc258dbac None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:40.499920 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5218903-54ba-45af-a1d1-d5bbc258dbac None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:40.500202 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5218903-54ba-45af-a1d1-d5bbc258dbac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:40.530753 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:40.566100 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b6c46f-23ee-4f35-b95d-e1b1f5075c09 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:40.568608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5218903-54ba-45af-a1d1-d5bbc258dbac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:40.570134 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5218903-54ba-45af-a1d1-d5bbc258dbac None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:40.600237 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6386e40-47eb-40d5-ac8c-aa363c908ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:40.601264 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6386e40-47eb-40d5-ac8c-aa363c908ec1 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:40.601545 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6386e40-47eb-40d5-ac8c-aa363c908ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:40.601850 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6386e40-47eb-40d5-ac8c-aa363c908ec1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:40.603033 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6386e40-47eb-40d5-ac8c-aa363c908ec1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:40.604525 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6386e40-47eb-40d5-ac8c-aa363c908ec1 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:40.606154 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 45/83] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:33:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:40.606915 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6386e40-47eb-40d5-ac8c-aa363c908ec1 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:40.658769 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:40.669667 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 39/84] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:33:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:33:40.670529 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5218903-54ba-45af-a1d1-d5bbc258dbac None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:40.709542 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6386e40-47eb-40d5-ac8c-aa363c908ec1 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:40.733606 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:40.744686 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c12ca-b6f4-466c-a261-81052742c53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:40.745365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c12ca-b6f4-466c-a261-81052742c53c None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:40.745687 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c12ca-b6f4-466c-a261-81052742c53c None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:40.745965 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c12ca-b6f4-466c-a261-81052742c53c None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:40.746730 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c12ca-b6f4-466c-a261-81052742c53c None admin] RBAC: Adding query filter params (name=member) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:40.747031 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c12ca-b6f4-466c-a261-81052742c53c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:40.748473 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c12ca-b6f4-466c-a261-81052742c53c None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:40.757104 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 46/85] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jul 20 14:33:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:40.758050 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c12ca-b6f4-466c-a261-81052742c53c None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:40.777741 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5218903-54ba-45af-a1d1-d5bbc258dbac None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:40.813061 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca7c1a-ed13-4aa9-901e-5b15206c9e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:40.813942 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca7c1a-ed13-4aa9-901e-5b15206c9e73 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:40.814233 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca7c1a-ed13-4aa9-901e-5b15206c9e73 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:40.814685 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca7c1a-ed13-4aa9-901e-5b15206c9e73 None admin] PATH_INFO: `/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:40.815983 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca7c1a-ed13-4aa9-901e-5b15206c9e73 None admin] RBAC: Authorizing `identity:get_project(project_id=e27e1edc913f4b99a1b0be6ab9d16b6f)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:40.817455 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca7c1a-ed13-4aa9-901e-5b15206c9e73 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:40.819243 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 40/86] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:33:40 2022] GET /identity/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:40.819993 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca7c1a-ed13-4aa9-901e-5b15206c9e73 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:40.860704 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c12ca-b6f4-466c-a261-81052742c53c None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:40.874884 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:40.895337 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d6851-e2ec-41b3-b71a-72f670746869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:40.895999 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d6851-e2ec-41b3-b71a-72f670746869 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:40.896337 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d6851-e2ec-41b3-b71a-72f670746869 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:40.896617 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d6851-e2ec-41b3-b71a-72f670746869 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:40.918271 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca7c1a-ed13-4aa9-901e-5b15206c9e73 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:40.947471 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:40.952965 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf2958-563b-4590-8506-34378af46813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:40.953666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf2958-563b-4590-8506-34378af46813 None admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:40.953955 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf2958-563b-4590-8506-34378af46813 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:40.954235 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf2958-563b-4590-8506-34378af46813 None admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:40.954981 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebf2958-563b-4590-8506-34378af46813 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:40.956487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebf2958-563b-4590-8506-34378af46813 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:40.964624 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d6851-e2ec-41b3-b71a-72f670746869 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:40.966114 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d6851-e2ec-41b3-b71a-72f670746869 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:40.977858 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 41/87] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jul 20 14:33:40 2022] POST /identity/v3/users => generated 289 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:33:40.978686 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf2958-563b-4590-8506-34378af46813 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:41.081181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 47/88] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:33:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:33:41.081990 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d6851-e2ec-41b3-b71a-72f670746869 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:41.095823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf2958-563b-4590-8506-34378af46813 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:41.099780 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:41.134896 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb6df61-ba66-4bbd-b5c7-61c29c72fb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:41.135847 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6df61-ba66-4bbd-b5c7-61c29c72fb6b None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:41.136119 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6df61-ba66-4bbd-b5c7-61c29c72fb6b None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:41.136391 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6df61-ba66-4bbd-b5c7-61c29c72fb6b None admin] PATH_INFO: `/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:41.137714 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb6df61-ba66-4bbd-b5c7-61c29c72fb6b None admin] RBAC: Authorizing `identity:get_project(project_id=e27e1edc913f4b99a1b0be6ab9d16b6f)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:41.139145 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb6df61-ba66-4bbd-b5c7-61c29c72fb6b None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:41.140904 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 42/89] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:33:40 2022] GET /identity/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:41.141664 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb6df61-ba66-4bbd-b5c7-61c29c72fb6b None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:41.208275 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:41.213346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d6851-e2ec-41b3-b71a-72f670746869 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:41.260859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f54787-6086-455e-8bdb-631d92141d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:41.262118 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f54787-6086-455e-8bdb-631d92141d6e None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:41.262507 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f54787-6086-455e-8bdb-631d92141d6e None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:41.262899 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f54787-6086-455e-8bdb-631d92141d6e None admin] PATH_INFO: `/v3/users/55e7a5b2a35b46a0a3eedafbe6c53a2c` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:41.264813 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f54787-6086-455e-8bdb-631d92141d6e None admin] RBAC: Authorizing `identity:get_user(user_id=55e7a5b2a35b46a0a3eedafbe6c53a2c)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:41.266476 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f54787-6086-455e-8bdb-631d92141d6e None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:41.269012 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 48/90] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:33:41 2022] GET /identity/v3/users/55e7a5b2a35b46a0a3eedafbe6c53a2c => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:41.270615 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb6df61-ba66-4bbd-b5c7-61c29c72fb6b None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:41.274461 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f54787-6086-455e-8bdb-631d92141d6e None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:41.317138 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2376f-73d1-4230-8de5-c64bfb3e97d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:41.317822 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2376f-73d1-4230-8de5-c64bfb3e97d3 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:41.318111 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2376f-73d1-4230-8de5-c64bfb3e97d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:41.318403 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2376f-73d1-4230-8de5-c64bfb3e97d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:41.319294 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2376f-73d1-4230-8de5-c64bfb3e97d3 None admin] RBAC: Adding query filter params (role.id=49bd5abf255048a0abaa9538e309f516, scope.project.id=e27e1edc913f4b99a1b0be6ab9d16b6f, user.id=651b2280231048d59d156e569aa33194) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:41.319614 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2376f-73d1-4230-8de5-c64bfb3e97d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:41.321241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2376f-73d1-4230-8de5-c64bfb3e97d3 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:41.330990 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 43/91] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:33:41 2022] GET /identity/v3/role_assignments?user.id=651b2280231048d59d156e569aa33194&scope.project.id=e27e1edc913f4b99a1b0be6ab9d16b6f&role.id=49bd5abf255048a0abaa9538e309f516 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:41.331936 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2376f-73d1-4230-8de5-c64bfb3e97d3 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:41.391618 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:41.439275 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f54787-6086-455e-8bdb-631d92141d6e None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:41.446865 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2376f-73d1-4230-8de5-c64bfb3e97d3 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:41.469182 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:41.500358 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba20b883-c53d-449f-9941-07ff3bb1cc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:41.501307 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20b883-c53d-449f-9941-07ff3bb1cc99 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:41.502537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20b883-c53d-449f-9941-07ff3bb1cc99 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:41.502983 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20b883-c53d-449f-9941-07ff3bb1cc99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:41.504707 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba20b883-c53d-449f-9941-07ff3bb1cc99 None admin] RBAC: Adding query filter params (role.id=7e394dd183b147ffa15b617f076bbb76, scope.project.id=e27e1edc913f4b99a1b0be6ab9d16b6f, user.id=55e7a5b2a35b46a0a3eedafbe6c53a2c) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:41.505346 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba20b883-c53d-449f-9941-07ff3bb1cc99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:41.508171 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba20b883-c53d-449f-9941-07ff3bb1cc99 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:41.515405 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 44/92] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:33:41 2022] GET /identity/v3/role_assignments?user.id=55e7a5b2a35b46a0a3eedafbe6c53a2c&scope.project.id=e27e1edc913f4b99a1b0be6ab9d16b6f&role.id=7e394dd183b147ffa15b617f076bbb76 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:41.516479 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba20b883-c53d-449f-9941-07ff3bb1cc99 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:41.528349 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c915df8-140d-47b1-911e-78103ca1bed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:41.529260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c915df8-140d-47b1-911e-78103ca1bed8 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:41.529553 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c915df8-140d-47b1-911e-78103ca1bed8 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:41.529842 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c915df8-140d-47b1-911e-78103ca1bed8 None admin] PATH_INFO: `/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:41.532155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c915df8-140d-47b1-911e-78103ca1bed8 None admin] RBAC: Authorizing `identity:get_project(project_id=e27e1edc913f4b99a1b0be6ab9d16b6f)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:41.534407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c915df8-140d-47b1-911e-78103ca1bed8 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:41.537113 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 49/93] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:33:41 2022] GET /identity/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f => generated 298 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:41.543276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c915df8-140d-47b1-911e-78103ca1bed8 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:41.727833 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba20b883-c53d-449f-9941-07ff3bb1cc99 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:41.774490 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c915df8-140d-47b1-911e-78103ca1bed8 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:41.785797 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62557407-2a4b-4ae0-a873-535e4c0440f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:41.786499 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62557407-2a4b-4ae0-a873-535e4c0440f7 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:41.786789 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62557407-2a4b-4ae0-a873-535e4c0440f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:41.787100 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62557407-2a4b-4ae0-a873-535e4c0440f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:41.853239 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5632c1-bedf-454d-b2a8-16e2ae85f71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:41.853984 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5632c1-bedf-454d-b2a8-16e2ae85f71b None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:41.854270 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5632c1-bedf-454d-b2a8-16e2ae85f71b None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:41.854598 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5632c1-bedf-454d-b2a8-16e2ae85f71b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:41.855491 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5632c1-bedf-454d-b2a8-16e2ae85f71b None admin] RBAC: Adding query filter params (role.id=b74189174ce24c1382712858dc35f3ce, scope.project.id=e27e1edc913f4b99a1b0be6ab9d16b6f, user.id=55e7a5b2a35b46a0a3eedafbe6c53a2c) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:41.855813 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5632c1-bedf-454d-b2a8-16e2ae85f71b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:41.857300 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5632c1-bedf-454d-b2a8-16e2ae85f71b None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:41.876992 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 50/94] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:33:41 2022] GET /identity/v3/role_assignments?user.id=55e7a5b2a35b46a0a3eedafbe6c53a2c&scope.project.id=e27e1edc913f4b99a1b0be6ab9d16b6f&role.id=b74189174ce24c1382712858dc35f3ce => generated 259 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:41.877370 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:41.904185 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62557407-2a4b-4ae0-a873-535e4c0440f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:41.905750 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62557407-2a4b-4ae0-a873-535e4c0440f7 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:42.030688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 45/95] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:33:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:33:42.037045 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5632c1-bedf-454d-b2a8-16e2ae85f71b None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:42.055629 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:42.148080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5632c1-bedf-454d-b2a8-16e2ae85f71b None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:42.185561 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a5581-505f-4aa5-ba49-b2eb80d4a1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:42.186471 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a5581-505f-4aa5-ba49-b2eb80d4a1a3 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:42.186757 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a5581-505f-4aa5-ba49-b2eb80d4a1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:42.187038 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a5581-505f-4aa5-ba49-b2eb80d4a1a3 None admin] PATH_INFO: `/v3/projects/21b6e12113924026b1b402ed414f7518` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:42.188660 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04a5581-505f-4aa5-ba49-b2eb80d4a1a3 None admin] RBAC: Authorizing `identity:get_project(project_id=21b6e12113924026b1b402ed414f7518)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:42.190233 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04a5581-505f-4aa5-ba49-b2eb80d4a1a3 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:42.192405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 51/96] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:33:42 2022] GET /identity/v3/projects/21b6e12113924026b1b402ed414f7518 => generated 312 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:42.200296 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62557407-2a4b-4ae0-a873-535e4c0440f7 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:42.309831 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:42.326227 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62557407-2a4b-4ae0-a873-535e4c0440f7 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:42.361934 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260e5fd-5c90-48b9-92f3-e701f0e5435f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:42.362615 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260e5fd-5c90-48b9-92f3-e701f0e5435f None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:42.362906 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260e5fd-5c90-48b9-92f3-e701f0e5435f None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:42.363224 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260e5fd-5c90-48b9-92f3-e701f0e5435f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:42.364081 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9260e5fd-5c90-48b9-92f3-e701f0e5435f None admin] RBAC: Adding query filter params (role.id=7e394dd183b147ffa15b617f076bbb76, scope.project.id=21b6e12113924026b1b402ed414f7518, user.id=55e7a5b2a35b46a0a3eedafbe6c53a2c) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:42.364408 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9260e5fd-5c90-48b9-92f3-e701f0e5435f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:42.365867 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9260e5fd-5c90-48b9-92f3-e701f0e5435f None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:42.371946 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 46/97] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:33:42 2022] GET /identity/v3/role_assignments?user.id=55e7a5b2a35b46a0a3eedafbe6c53a2c&scope.project.id=21b6e12113924026b1b402ed414f7518&role.id=7e394dd183b147ffa15b617f076bbb76 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:43.787525 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42caf86b-7119-4133-a176-dfecc35fc305 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:43.788516 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42caf86b-7119-4133-a176-dfecc35fc305 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:43.789011 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42caf86b-7119-4133-a176-dfecc35fc305 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:43.790673 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 52/98] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:43.796295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992e293-5974-419e-b73a-3d6cca7b1ad4 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:43.796735 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992e293-5974-419e-b73a-3d6cca7b1ad4 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:43.796917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992e293-5974-419e-b73a-3d6cca7b1ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:43.819595 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-b992e293-5974-419e-b73a-3d6cca7b1ad4 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:43.820869 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-b992e293-5974-419e-b73a-3d6cca7b1ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:43.918410 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 47/99] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 14:33:43.964610 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42caf86b-7119-4133-a176-dfecc35fc305 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:44.062690 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:44.086288 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42caf86b-7119-4133-a176-dfecc35fc305 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:44.125027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14eca3c-5842-400e-8846-940a567a03e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:44.126217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14eca3c-5842-400e-8846-940a567a03e4 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:44.126733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14eca3c-5842-400e-8846-940a567a03e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:44.127289 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14eca3c-5842-400e-8846-940a567a03e4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:44.138740 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14eca3c-5842-400e-8846-940a567a03e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:33:44.146950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-f14eca3c-5842-400e-8846-940a567a03e4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 14:33:44.146950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:33:44.146950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:33:44.146950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:33:44.146950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:33:44.146950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:33:44.146950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:33:44.146950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:33:44.146950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:33:44.146950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:33:44.146950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:33:44.146950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:33:44.146950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:33:44.146950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:33:44.146950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:33:44.146950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:33:44.147834 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:33:44.147834 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:33:44.147834 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:33:44.147834 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:33:44.147834 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:33:44.147834 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:33:44.147834 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:33:44.147834 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:33:44.147834 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:33:44.147834 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:33:44.147834 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:33:44.147834 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:33:44.147834 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:33:44.147834 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:33:44.147834 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:33:44.147834 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:33:44.148652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:33:44.148652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:33:44.148652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:33:44.148652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:33:44.148652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:33:44.148652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:33:44.148652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:33:44.148652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:33:44.148652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:33:44.148652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:33:44.148652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 14:33:44.148652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:33:44.151380 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 53/100] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:33:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:33:44.158761 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b992e293-5974-419e-b73a-3d6cca7b1ad4 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:44.294948 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:44.326742 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b992e293-5974-419e-b73a-3d6cca7b1ad4 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:44.337439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48972d8a-ec1f-45f8-8c4f-157a2f67b835 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:44.338130 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48972d8a-ec1f-45f8-8c4f-157a2f67b835 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:44.338624 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48972d8a-ec1f-45f8-8c4f-157a2f67b835 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:44.340279 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 54/101] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 14:33:44.345787 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3e139-7cfa-4cda-8256-094260cfe400 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:44.346355 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3e139-7cfa-4cda-8256-094260cfe400 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:44.346836 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3e139-7cfa-4cda-8256-094260cfe400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:44.377201 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-b7a3e139-7cfa-4cda-8256-094260cfe400 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:44.378814 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-b7a3e139-7cfa-4cda-8256-094260cfe400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:44.384737 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afba31-e668-4efe-9b8c-ba77fc011c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:44.385457 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afba31-e668-4efe-9b8c-ba77fc011c27 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:44.385749 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afba31-e668-4efe-9b8c-ba77fc011c27 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:44.391072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afba31-e668-4efe-9b8c-ba77fc011c27 None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:44.391072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3afba31-e668-4efe-9b8c-ba77fc011c27 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:44.391072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3afba31-e668-4efe-9b8c-ba77fc011c27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:44.391072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3afba31-e668-4efe-9b8c-ba77fc011c27 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:44.407182 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 48/102] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jul 20 14:33:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:44.412224 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afba31-e668-4efe-9b8c-ba77fc011c27 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:44.480530 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 55/103] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 14:33:44.486225 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02561979-b4ae-4fa1-80db-a11d40a84785 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:44.486615 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02561979-b4ae-4fa1-80db-a11d40a84785 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:44.486807 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02561979-b4ae-4fa1-80db-a11d40a84785 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:44.488444 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 56/104] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:44.496218 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e6522-0d47-48c9-b7f7-ec20c4d8fdd4 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:44.496689 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e6522-0d47-48c9-b7f7-ec20c4d8fdd4 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:44.497115 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e6522-0d47-48c9-b7f7-ec20c4d8fdd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:44.528266 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-366e6522-0d47-48c9-b7f7-ec20c4d8fdd4 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:44.529364 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-366e6522-0d47-48c9-b7f7-ec20c4d8fdd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:44.623151 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afba31-e668-4efe-9b8c-ba77fc011c27 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:44.659836 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6050b-6cd9-4d50-813e-027f6f0b2c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:44.661016 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6050b-6cd9-4d50-813e-027f6f0b2c7f None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:44.661616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6050b-6cd9-4d50-813e-027f6f0b2c7f None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:44.661911 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6050b-6cd9-4d50-813e-027f6f0b2c7f None admin] PATH_INFO: `/v3/users/11ef2269904344a0b9df24ec7510daf6` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:44.673881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:44.676474 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 57/105] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:33:44.677393 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e6522-0d47-48c9-b7f7-ec20c4d8fdd4 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:44.678530 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6050b-6cd9-4d50-813e-027f6f0b2c7f None admin] RBAC: Authorizing `identity:get_user(user_id=11ef2269904344a0b9df24ec7510daf6)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:44.680166 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:44.681500 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6050b-6cd9-4d50-813e-027f6f0b2c7f None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:44.685031 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 49/106] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:33:44 2022] GET /identity/v3/users/11ef2269904344a0b9df24ec7510daf6 => generated 289 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:33:44.690925 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6050b-6cd9-4d50-813e-027f6f0b2c7f None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:44.841780 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6050b-6cd9-4d50-813e-027f6f0b2c7f None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:44.878525 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e6522-0d47-48c9-b7f7-ec20c4d8fdd4 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:44.894853 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee14c7-ac6d-42ca-8cbd-531d08994146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:44.896024 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee14c7-ac6d-42ca-8cbd-531d08994146 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:44.896318 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee14c7-ac6d-42ca-8cbd-531d08994146 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:44.896602 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee14c7-ac6d-42ca-8cbd-531d08994146 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:44.917762 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c24e98-993e-4fc5-ae45-5b8450bf5410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:44.918903 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c24e98-993e-4fc5-ae45-5b8450bf5410 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:44.919386 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c24e98-993e-4fc5-ae45-5b8450bf5410 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:44.919935 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c24e98-993e-4fc5-ae45-5b8450bf5410 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:44.921449 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c24e98-993e-4fc5-ae45-5b8450bf5410 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:44.923163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c24e98-993e-4fc5-ae45-5b8450bf5410 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:44.925457 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 58/107] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:33:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:44.928711 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c24e98-993e-4fc5-ae45-5b8450bf5410 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:44.973257 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:44.977402 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee14c7-ac6d-42ca-8cbd-531d08994146 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:44.978924 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee14c7-ac6d-42ca-8cbd-531d08994146 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:44.999777 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:45.030746 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c24e98-993e-4fc5-ae45-5b8450bf5410 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:45.066604 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efd335-45b8-4025-81c8-6cef08dcd53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:45.067454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efd335-45b8-4025-81c8-6cef08dcd53d None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:45.067740 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efd335-45b8-4025-81c8-6cef08dcd53d None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:45.068025 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efd335-45b8-4025-81c8-6cef08dcd53d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:45.078921 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8efd335-45b8-4025-81c8-6cef08dcd53d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:33:45.086679 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-b8efd335-45b8-4025-81c8-6cef08dcd53d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:33:45.086679 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:33:45.086679 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:33:45.086679 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:33:45.086679 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:33:45.086679 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:33:45.086679 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:33:45.086679 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:33:45.086679 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:33:45.086679 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:33:45.086679 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:33:45.086679 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:33:45.086679 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:33:45.086679 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:33:45.086679 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:33:45.086679 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:33:45.087560 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:33:45.087560 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:33:45.087560 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:33:45.087560 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:33:45.087560 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:33:45.087560 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:33:45.087560 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:33:45.087560 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:33:45.087560 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:33:45.087560 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:33:45.087560 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:33:45.087560 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:33:45.087560 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:33:45.087560 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:33:45.087560 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:33:45.087560 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:33:45.088417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:33:45.088417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:33:45.088417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:33:45.088417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:33:45.088417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:33:45.088417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:33:45.088417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:33:45.088417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:33:45.088417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:33:45.088417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:33:45.088417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:33:45.088417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:33:45.089496 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 50/108] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:33:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:33:45.090453 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e6953-4a6a-4d85-a198-9304a1fd678f None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:45.090748 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e6953-4a6a-4d85-a198-9304a1fd678f None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:45.091028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e6953-4a6a-4d85-a198-9304a1fd678f None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:45.095339 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 51/109] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:45.095463 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 59/110] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:33:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:33:45.095558 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33bf20-bcc1-48f6-85fe-8100480443b3 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:45.095558 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33bf20-bcc1-48f6-85fe-8100480443b3 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:45.095688 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efd335-45b8-4025-81c8-6cef08dcd53d None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:45.095755 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33bf20-bcc1-48f6-85fe-8100480443b3 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:45.095755 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 52/111] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:45.100366 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33bf20-bcc1-48f6-85fe-8100480443b3 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:45.202553 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efd335-45b8-4025-81c8-6cef08dcd53d None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:45.204406 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:45.211338 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33bf20-bcc1-48f6-85fe-8100480443b3 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:45.232588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:45.239603 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c5c8c-fccb-469d-b20a-3326a96f8b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:45.240286 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c5c8c-fccb-469d-b20a-3326a96f8b37 None admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:45.240571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c5c8c-fccb-469d-b20a-3326a96f8b37 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:45.240849 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c5c8c-fccb-469d-b20a-3326a96f8b37 None admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:45.241577 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966c5c8c-fccb-469d-b20a-3326a96f8b37 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:45.243192 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966c5c8c-fccb-469d-b20a-3326a96f8b37 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:45.247672 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77cd4e1-526e-4298-8a7f-b4a98a413a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:45.248990 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77cd4e1-526e-4298-8a7f-b4a98a413a3e None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:45.249276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77cd4e1-526e-4298-8a7f-b4a98a413a3e None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:45.249558 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77cd4e1-526e-4298-8a7f-b4a98a413a3e None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:45.250333 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77cd4e1-526e-4298-8a7f-b4a98a413a3e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:45.250647 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77cd4e1-526e-4298-8a7f-b4a98a413a3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:45.252078 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77cd4e1-526e-4298-8a7f-b4a98a413a3e None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:45.260825 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 53/112] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:33:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:45.261640 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77cd4e1-526e-4298-8a7f-b4a98a413a3e None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:45.275075 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 60/113] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Jul 20 14:33:45 2022] POST /identity/v3/projects => generated 302 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:33:45.275314 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a313c2-9309-4cd5-ab7b-e2f52d7da9f6 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:45.275604 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a313c2-9309-4cd5-ab7b-e2f52d7da9f6 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:45.275883 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a313c2-9309-4cd5-ab7b-e2f52d7da9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:45.294502 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-05a313c2-9309-4cd5-ab7b-e2f52d7da9f6 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:45.296151 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-05a313c2-9309-4cd5-ab7b-e2f52d7da9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:45.371128 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 61/114] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:33:45.380716 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54dd6dc-01ad-4a1d-9bea-06ed86b7ed1d None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:45.381453 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54dd6dc-01ad-4a1d-9bea-06ed86b7ed1d None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:45.381749 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54dd6dc-01ad-4a1d-9bea-06ed86b7ed1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:45.385680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77cd4e1-526e-4298-8a7f-b4a98a413a3e None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:45.401144 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-b54dd6dc-01ad-4a1d-9bea-06ed86b7ed1d None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:45.402518 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-b54dd6dc-01ad-4a1d-9bea-06ed86b7ed1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:45.422969 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c74943-e6c0-4f9b-a46f-fb452289488b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:45.423951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c74943-e6c0-4f9b-a46f-fb452289488b None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:45.424602 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c74943-e6c0-4f9b-a46f-fb452289488b None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:45.424890 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c74943-e6c0-4f9b-a46f-fb452289488b None admin] PATH_INFO: `/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:45.426145 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c74943-e6c0-4f9b-a46f-fb452289488b None admin] RBAC: Authorizing `identity:get_project(project_id=e27e1edc913f4b99a1b0be6ab9d16b6f)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:45.427676 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c74943-e6c0-4f9b-a46f-fb452289488b None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:45.431237 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 54/115] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:33:45 2022] GET /identity/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:45.431237 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789bdce-ca8a-4b93-ad9a-89e73780879a None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:45.431237 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789bdce-ca8a-4b93-ad9a-89e73780879a None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:45.431237 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789bdce-ca8a-4b93-ad9a-89e73780879a None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:45.432616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 55/116] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:45.433135 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789bdce-ca8a-4b93-ad9a-89e73780879a None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:45.463070 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:45.483537 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 62/117] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:33:45.487194 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54dd6dc-01ad-4a1d-9bea-06ed86b7ed1d None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:45.532173 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:45.548884 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789bdce-ca8a-4b93-ad9a-89e73780879a None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:45.584548 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd7f7c6-defd-4fce-83f2-e5cb3b083d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:45.585436 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7f7c6-defd-4fce-83f2-e5cb3b083d83 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:45.585728 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7f7c6-defd-4fce-83f2-e5cb3b083d83 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:45.586044 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7f7c6-defd-4fce-83f2-e5cb3b083d83 None admin] PATH_INFO: `/v3/users/651b2280231048d59d156e569aa33194` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:45.587311 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd7f7c6-defd-4fce-83f2-e5cb3b083d83 None admin] RBAC: Authorizing `identity:get_user(user_id=651b2280231048d59d156e569aa33194)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:45.588798 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd7f7c6-defd-4fce-83f2-e5cb3b083d83 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:45.594814 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 56/118] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:33:45 2022] GET /identity/v3/users/651b2280231048d59d156e569aa33194 => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:45.594814 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd7f7c6-defd-4fce-83f2-e5cb3b083d83 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:45.600951 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54dd6dc-01ad-4a1d-9bea-06ed86b7ed1d None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:45.637738 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0989d5-deb7-415e-b0ed-bd3573328db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:45.638599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0989d5-deb7-415e-b0ed-bd3573328db1 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:45.638917 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0989d5-deb7-415e-b0ed-bd3573328db1 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:45.639242 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0989d5-deb7-415e-b0ed-bd3573328db1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:45.650011 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0989d5-deb7-415e-b0ed-bd3573328db1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:33:45.658076 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-8c0989d5-deb7-415e-b0ed-bd3573328db1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 14:33:45.658076 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:33:45.658076 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:33:45.658076 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:33:45.658076 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:33:45.658076 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:33:45.658076 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:33:45.658076 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:33:45.658076 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:33:45.658076 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:33:45.658076 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:33:45.658076 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:33:45.658076 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:33:45.658076 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:33:45.658076 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:33:45.658076 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:33:45.659010 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:33:45.659010 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:33:45.659010 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:33:45.659010 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:33:45.659010 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:33:45.659010 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:33:45.659010 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:33:45.659010 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:33:45.659010 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:33:45.659010 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:33:45.659010 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:33:45.659010 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:33:45.659010 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:33:45.659010 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:33:45.659010 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:33:45.659010 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:33:45.665302 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:33:45.665302 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:33:45.665302 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:33:45.665302 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:33:45.665302 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:33:45.665302 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:33:45.665302 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:33:45.665302 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:33:45.665302 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:33:45.665302 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:33:45.665302 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 14:33:45.665302 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:33:45.665302 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 63/119] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jul 20 14:33:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:33:45.665302 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68622917-65ee-4c6f-a6c8-b423a8c334b3 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:45.665977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68622917-65ee-4c6f-a6c8-b423a8c334b3 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:45.665977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68622917-65ee-4c6f-a6c8-b423a8c334b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:45.678465 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-68622917-65ee-4c6f-a6c8-b423a8c334b3 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:45.679732 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-68622917-65ee-4c6f-a6c8-b423a8c334b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:45.704120 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:45.710371 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd7f7c6-defd-4fce-83f2-e5cb3b083d83 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:45.747201 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c067558-f27d-402f-8f89-0162cae1087f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:45.747802 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c067558-f27d-402f-8f89-0162cae1087f None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:45.748095 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c067558-f27d-402f-8f89-0162cae1087f None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:45.748383 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c067558-f27d-402f-8f89-0162cae1087f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:45.749260 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c067558-f27d-402f-8f89-0162cae1087f None admin] RBAC: Adding query filter params (role.id=483f28cfc7444513a119d5d363f64518, scope.project.id=e27e1edc913f4b99a1b0be6ab9d16b6f, user.id=11ef2269904344a0b9df24ec7510daf6) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:45.749363 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 64/120] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:33:45.749855 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c067558-f27d-402f-8f89-0162cae1087f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:45.750332 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68622917-65ee-4c6f-a6c8-b423a8c334b3 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:45.751393 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c067558-f27d-402f-8f89-0162cae1087f None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:45.763177 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 57/121] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:33:45 2022] GET /identity/v3/role_assignments?user.id=11ef2269904344a0b9df24ec7510daf6&scope.project.id=e27e1edc913f4b99a1b0be6ab9d16b6f&role.id=483f28cfc7444513a119d5d363f64518 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:45.763177 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c067558-f27d-402f-8f89-0162cae1087f None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:45.797860 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:45.901505 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68622917-65ee-4c6f-a6c8-b423a8c334b3 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:45.906111 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c067558-f27d-402f-8f89-0162cae1087f None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:45.948359 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c8c51-603a-42f9-9c60-f3b9db41f3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:45.949191 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c8c51-603a-42f9-9c60-f3b9db41f3a3 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:45.949480 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c8c51-603a-42f9-9c60-f3b9db41f3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:45.949692 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b5c80-ae7a-4439-83ba-27f30f45c008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:45.949816 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c8c51-603a-42f9-9c60-f3b9db41f3a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:45.950742 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b5c80-ae7a-4439-83ba-27f30f45c008 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:45.951185 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b5c80-ae7a-4439-83ba-27f30f45c008 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:45.951824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b5c80-ae7a-4439-83ba-27f30f45c008 None admin] PATH_INFO: `/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:45.953497 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b5c80-ae7a-4439-83ba-27f30f45c008 None admin] RBAC: Authorizing `identity:get_project(project_id=e27e1edc913f4b99a1b0be6ab9d16b6f)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:45.955389 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b5c80-ae7a-4439-83ba-27f30f45c008 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:45.959210 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 58/122] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:33:45 2022] GET /identity/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:45.960045 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b5c80-ae7a-4439-83ba-27f30f45c008 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:45.966854 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c8c51-603a-42f9-9c60-f3b9db41f3a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:33:45.976745 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-fd4c8c51-603a-42f9-9c60-f3b9db41f3a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:33:45.976745 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:33:45.976745 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:33:45.976745 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:33:45.976745 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:33:45.976745 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:33:45.976745 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:33:45.976745 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:33:45.976745 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:33:45.976745 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:33:45.976745 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:33:45.976745 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:33:45.976745 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:33:45.976745 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:33:45.976745 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:33:45.976745 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:33:45.978503 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:33:45.978503 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:33:45.978503 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:33:45.978503 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:33:45.978503 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:33:45.978503 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:33:45.978503 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:33:45.978503 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:33:45.978503 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:33:45.978503 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:33:45.978503 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:33:45.978503 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:33:45.978503 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:33:45.978503 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:33:45.978503 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:33:45.978503 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:33:45.979696 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:33:45.979696 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:33:45.979696 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:33:45.979696 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:33:45.979696 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:33:45.979696 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:33:45.979696 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:33:45.979696 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:33:45.979696 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:33:45.979696 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:33:45.979696 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:33:45.979696 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:33:45.979696 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 65/123] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:33:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:33:45.980462 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c8c51-603a-42f9-9c60-f3b9db41f3a3 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:45.997126 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:46.096082 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b5c80-ae7a-4439-83ba-27f30f45c008 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:46.100325 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:46.111564 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c8c51-603a-42f9-9c60-f3b9db41f3a3 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:46.147337 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab66f7a-42d7-48f4-ac05-b5c7a2041a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:46.148195 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab66f7a-42d7-48f4-ac05-b5c7a2041a8f None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:46.148609 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab66f7a-42d7-48f4-ac05-b5c7a2041a8f None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:46.148981 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab66f7a-42d7-48f4-ac05-b5c7a2041a8f None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:46.149761 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc7b8d-6e12-463a-8800-30d1c862558c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:46.150185 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab66f7a-42d7-48f4-ac05-b5c7a2041a8f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:46.150591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab66f7a-42d7-48f4-ac05-b5c7a2041a8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:46.151026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc7b8d-6e12-463a-8800-30d1c862558c None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:46.151488 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc7b8d-6e12-463a-8800-30d1c862558c None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:46.151942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc7b8d-6e12-463a-8800-30d1c862558c None admin] PATH_INFO: `/v3/roles/member` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:46.152642 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab66f7a-42d7-48f4-ac05-b5c7a2041a8f None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:46.162503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 59/124] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:33:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:46.166141 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbc7b8d-6e12-463a-8800-30d1c862558c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:33:46.168097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab66f7a-42d7-48f4-ac05-b5c7a2041a8f None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:46.176407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-3dbc7b8d-6e12-463a-8800-30d1c862558c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:33:46.176407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:33:46.176407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:33:46.176407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:33:46.176407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:33:46.176407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:33:46.176407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:33:46.176407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:33:46.176407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:33:46.176407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:33:46.176407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:33:46.176407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:33:46.176407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:33:46.176407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:33:46.176407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:33:46.176407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:33:46.177566 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:33:46.177566 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:33:46.177566 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:33:46.177566 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:33:46.177566 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:33:46.177566 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:33:46.177566 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:33:46.177566 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:33:46.177566 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:33:46.177566 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:33:46.177566 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:33:46.177566 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:33:46.177566 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:33:46.177566 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:33:46.177566 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:33:46.177566 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:33:46.178556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:33:46.178556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:33:46.178556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:33:46.178556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:33:46.178556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:33:46.178556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:33:46.178556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:33:46.178556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:33:46.178556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:33:46.178556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:33:46.178556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:33:46.178556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:33:46.181238 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 66/125] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:33:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:33:46.182180 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc7b8d-6e12-463a-8800-30d1c862558c None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:46.296947 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:46.301424 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc7b8d-6e12-463a-8800-30d1c862558c None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:46.308415 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab66f7a-42d7-48f4-ac05-b5c7a2041a8f None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:46.343682 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f9096f-b88d-45b0-b7d6-71550500a836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:46.344867 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f9096f-b88d-45b0-b7d6-71550500a836 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:46.345572 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f9096f-b88d-45b0-b7d6-71550500a836 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:46.346329 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f9096f-b88d-45b0-b7d6-71550500a836 None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:46.347367 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f9096f-b88d-45b0-b7d6-71550500a836 None admin] RBAC: Adding query filter params (name=member) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:46.347939 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f9096f-b88d-45b0-b7d6-71550500a836 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:46.349692 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f9096f-b88d-45b0-b7d6-71550500a836 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:46.350313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ab4c9-e484-438e-89de-b04edabd384d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:46.351446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ab4c9-e484-438e-89de-b04edabd384d None admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:46.352030 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ab4c9-e484-438e-89de-b04edabd384d None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:46.352400 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ab4c9-e484-438e-89de-b04edabd384d None admin] PATH_INFO: `/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f/users/651b2280231048d59d156e569aa33194/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:46.354698 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667ab4c9-e484-438e-89de-b04edabd384d None admin] RBAC: Authorizing `identity:create_grant(project_id=e27e1edc913f4b99a1b0be6ab9d16b6f, user_id=651b2280231048d59d156e569aa33194, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:46.356312 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667ab4c9-e484-438e-89de-b04edabd384d None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:46.361091 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 67/126] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jul 20 14:33:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:46.362289 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f9096f-b88d-45b0-b7d6-71550500a836 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:46.376379 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 60/127] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jul 20 14:33:46 2022] PUT /identity/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f/users/651b2280231048d59d156e569aa33194/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:33:46.377612 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ab4c9-e484-438e-89de-b04edabd384d None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:46.379846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:46.504567 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f9096f-b88d-45b0-b7d6-71550500a836 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:46.544676 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28712ab9-925c-497b-8160-b5a6aed967b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:46.545677 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28712ab9-925c-497b-8160-b5a6aed967b8 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:46.545971 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28712ab9-925c-497b-8160-b5a6aed967b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:46.546252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28712ab9-925c-497b-8160-b5a6aed967b8 None admin] PATH_INFO: `/v3/users/55e7a5b2a35b46a0a3eedafbe6c53a2c` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:46.547574 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28712ab9-925c-497b-8160-b5a6aed967b8 None admin] RBAC: Authorizing `identity:get_user(user_id=55e7a5b2a35b46a0a3eedafbe6c53a2c)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:46.549449 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28712ab9-925c-497b-8160-b5a6aed967b8 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:46.551427 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 68/128] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:33:46 2022] GET /identity/v3/users/55e7a5b2a35b46a0a3eedafbe6c53a2c => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:46.552093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28712ab9-925c-497b-8160-b5a6aed967b8 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:46.630882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ab4c9-e484-438e-89de-b04edabd384d None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:46.646324 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:46.668005 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28712ab9-925c-497b-8160-b5a6aed967b8 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:46.681606 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb99b4-0503-473f-b738-f1bc54cba26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:46.682674 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb99b4-0503-473f-b738-f1bc54cba26a None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:46.683427 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb99b4-0503-473f-b738-f1bc54cba26a None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:46.684363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb99b4-0503-473f-b738-f1bc54cba26a None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:46.685626 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb99b4-0503-473f-b738-f1bc54cba26a None admin] RBAC: Adding query filter params (name=member) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:46.686140 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb99b4-0503-473f-b738-f1bc54cba26a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:46.688383 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb99b4-0503-473f-b738-f1bc54cba26a None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:46.702941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 61/129] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jul 20 14:33:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:46.707896 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb99b4-0503-473f-b738-f1bc54cba26a None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:46.726319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:46.740054 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e77f86-1aa4-4f6c-b9f6-5a177cf77d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:46.741086 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e77f86-1aa4-4f6c-b9f6-5a177cf77d9e None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:46.741381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e77f86-1aa4-4f6c-b9f6-5a177cf77d9e None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:46.741663 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e77f86-1aa4-4f6c-b9f6-5a177cf77d9e None admin] PATH_INFO: `/v3/users/55e7a5b2a35b46a0a3eedafbe6c53a2c` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:46.746821 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e77f86-1aa4-4f6c-b9f6-5a177cf77d9e None admin] RBAC: Authorizing `identity:get_user(user_id=55e7a5b2a35b46a0a3eedafbe6c53a2c)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:46.748444 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e77f86-1aa4-4f6c-b9f6-5a177cf77d9e None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:46.750444 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 69/130] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:33:46 2022] GET /identity/v3/users/55e7a5b2a35b46a0a3eedafbe6c53a2c => generated 275 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:46.751286 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e77f86-1aa4-4f6c-b9f6-5a177cf77d9e None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:46.855289 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb99b4-0503-473f-b738-f1bc54cba26a None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:46.925222 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb2514-6b7f-4f72-a0e3-a989018462de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:46.926875 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb2514-6b7f-4f72-a0e3-a989018462de None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:46.927374 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb2514-6b7f-4f72-a0e3-a989018462de None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:46.927894 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb2514-6b7f-4f72-a0e3-a989018462de None admin] PATH_INFO: `/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:46.930392 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcb2514-6b7f-4f72-a0e3-a989018462de None admin] RBAC: Authorizing `identity:get_project(project_id=e27e1edc913f4b99a1b0be6ab9d16b6f)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:46.932516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcb2514-6b7f-4f72-a0e3-a989018462de None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:46.935592 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 62/131] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:33:46 2022] GET /identity/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:46.938355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb2514-6b7f-4f72-a0e3-a989018462de None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:46.949332 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e77f86-1aa4-4f6c-b9f6-5a177cf77d9e None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:46.999017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe929e84-b108-4d31-802f-2a080aa2c647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:47.000059 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe929e84-b108-4d31-802f-2a080aa2c647 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:47.000457 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe929e84-b108-4d31-802f-2a080aa2c647 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:47.000744 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe929e84-b108-4d31-802f-2a080aa2c647 None admin] PATH_INFO: `/v3/users/55e7a5b2a35b46a0a3eedafbe6c53a2c` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:47.003893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe929e84-b108-4d31-802f-2a080aa2c647 None admin] RBAC: Authorizing `identity:get_user(user_id=55e7a5b2a35b46a0a3eedafbe6c53a2c)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:47.005385 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe929e84-b108-4d31-802f-2a080aa2c647 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:47.007352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 70/132] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:33:46 2022] GET /identity/v3/users/55e7a5b2a35b46a0a3eedafbe6c53a2c => generated 275 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:47.008207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe929e84-b108-4d31-802f-2a080aa2c647 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:47.032835 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:47.056014 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb2514-6b7f-4f72-a0e3-a989018462de None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:47.068102 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:47.093808 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae23f8-f98e-4928-8191-bef5827a5cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:47.094788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae23f8-f98e-4928-8191-bef5827a5cde None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:47.095185 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae23f8-f98e-4928-8191-bef5827a5cde None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:47.095532 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae23f8-f98e-4928-8191-bef5827a5cde None admin] PATH_INFO: `/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:47.096810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae23f8-f98e-4928-8191-bef5827a5cde None admin] RBAC: Authorizing `identity:get_project(project_id=e27e1edc913f4b99a1b0be6ab9d16b6f)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:47.098340 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae23f8-f98e-4928-8191-bef5827a5cde None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:47.100469 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 63/133] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:33:46 2022] GET /identity/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:47.101317 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae23f8-f98e-4928-8191-bef5827a5cde None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:47.218642 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe929e84-b108-4d31-802f-2a080aa2c647 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:47.259432 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae23f8-f98e-4928-8191-bef5827a5cde None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:47.260324 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a116e7d-08fa-4e8b-957b-bbc6891ce268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:47.261484 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a116e7d-08fa-4e8b-957b-bbc6891ce268 None admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:47.261984 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a116e7d-08fa-4e8b-957b-bbc6891ce268 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:47.262354 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a116e7d-08fa-4e8b-957b-bbc6891ce268 None admin] PATH_INFO: `/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f/users/55e7a5b2a35b46a0a3eedafbe6c53a2c/roles/b74189174ce24c1382712858dc35f3ce` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:47.264836 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a116e7d-08fa-4e8b-957b-bbc6891ce268 None admin] RBAC: Authorizing `identity:create_grant(project_id=e27e1edc913f4b99a1b0be6ab9d16b6f, user_id=55e7a5b2a35b46a0a3eedafbe6c53a2c, role_id=b74189174ce24c1382712858dc35f3ce)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:47.266397 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a116e7d-08fa-4e8b-957b-bbc6891ce268 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:47.290671 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 71/134] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jul 20 14:33:47 2022] PUT /identity/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f/users/55e7a5b2a35b46a0a3eedafbe6c53a2c/roles/b74189174ce24c1382712858dc35f3ce => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:33:47.291673 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a116e7d-08fa-4e8b-957b-bbc6891ce268 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:47.321312 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b45b27-32c1-4ab9-bb1e-483545b2b30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:47.322384 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b45b27-32c1-4ab9-bb1e-483545b2b30e None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:47.322732 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b45b27-32c1-4ab9-bb1e-483545b2b30e None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:47.323097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b45b27-32c1-4ab9-bb1e-483545b2b30e None admin] PATH_INFO: `/v3/projects/21b6e12113924026b1b402ed414f7518` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:47.325765 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b45b27-32c1-4ab9-bb1e-483545b2b30e None admin] RBAC: Authorizing `identity:get_project(project_id=21b6e12113924026b1b402ed414f7518)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:47.328698 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b45b27-32c1-4ab9-bb1e-483545b2b30e None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:47.330717 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 64/135] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:33:47 2022] GET /identity/v3/projects/21b6e12113924026b1b402ed414f7518 => generated 312 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:47.345678 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b45b27-32c1-4ab9-bb1e-483545b2b30e None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:47.373261 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:47.459187 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:47.517150 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a116e7d-08fa-4e8b-957b-bbc6891ce268 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:47.526847 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b45b27-32c1-4ab9-bb1e-483545b2b30e None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:47.597174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f75d2-066f-4109-be70-cde701d1d01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:47.598125 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f75d2-066f-4109-be70-cde701d1d01b None admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:47.598416 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f75d2-066f-4109-be70-cde701d1d01b None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:47.598849 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f75d2-066f-4109-be70-cde701d1d01b None admin] PATH_INFO: `/v3/projects/21b6e12113924026b1b402ed414f7518/users/55e7a5b2a35b46a0a3eedafbe6c53a2c/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:47.604601 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f75d2-066f-4109-be70-cde701d1d01b None admin] RBAC: Authorizing `identity:create_grant(project_id=21b6e12113924026b1b402ed414f7518, user_id=55e7a5b2a35b46a0a3eedafbe6c53a2c, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:47.606093 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f75d2-066f-4109-be70-cde701d1d01b None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:47.607240 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bac94c-997e-4f82-b9d3-fa07b7427b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:47.608409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bac94c-997e-4f82-b9d3-fa07b7427b9f None admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:47.608901 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bac94c-997e-4f82-b9d3-fa07b7427b9f None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:47.609378 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bac94c-997e-4f82-b9d3-fa07b7427b9f None admin] PATH_INFO: `/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f/users/55e7a5b2a35b46a0a3eedafbe6c53a2c/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:47.611634 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bac94c-997e-4f82-b9d3-fa07b7427b9f None admin] RBAC: Authorizing `identity:create_grant(project_id=e27e1edc913f4b99a1b0be6ab9d16b6f, user_id=55e7a5b2a35b46a0a3eedafbe6c53a2c, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:47.613340 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bac94c-997e-4f82-b9d3-fa07b7427b9f None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:47.629419 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 65/136] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jul 20 14:33:47 2022] PUT /identity/v3/projects/21b6e12113924026b1b402ed414f7518/users/55e7a5b2a35b46a0a3eedafbe6c53a2c/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:33:47.647525 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 72/137] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jul 20 14:33:47 2022] PUT /identity/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f/users/55e7a5b2a35b46a0a3eedafbe6c53a2c/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:33:47.671002 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b9ed7-97d4-439b-a384-5f42d1cff522 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:47.671315 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b9ed7-97d4-439b-a384-5f42d1cff522 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:47.671601 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b9ed7-97d4-439b-a384-5f42d1cff522 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:47.673220 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 66/138] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:47.679835 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d95e3f-3403-4f31-9c15-af413157e485 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:47.680493 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d95e3f-3403-4f31-9c15-af413157e485 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:47.680996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d95e3f-3403-4f31-9c15-af413157e485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:47.701057 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-d5d95e3f-3403-4f31-9c15-af413157e485 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:47.702774 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-d5d95e3f-3403-4f31-9c15-af413157e485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:47.783357 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 73/139] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:33:47.808894 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b9ed7-97d4-439b-a384-5f42d1cff522 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:47.922915 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:47.952858 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b9ed7-97d4-439b-a384-5f42d1cff522 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:47.990985 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca6d74-fc51-4f66-8b5b-ff0204d81aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:47.991852 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca6d74-fc51-4f66-8b5b-ff0204d81aca None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:47.992295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca6d74-fc51-4f66-8b5b-ff0204d81aca None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:47.992582 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca6d74-fc51-4f66-8b5b-ff0204d81aca None admin] PATH_INFO: `/v3/domains/default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:47.993805 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ca6d74-fc51-4f66-8b5b-ff0204d81aca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:47.995470 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ca6d74-fc51-4f66-8b5b-ff0204d81aca None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:47.997216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 67/140] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:33:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:48.002828 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d95e3f-3403-4f31-9c15-af413157e485 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:48.073532 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:48.108279 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d95e3f-3403-4f31-9c15-af413157e485 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:48.155237 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bb2b0-470b-44e8-b57f-3a780948cb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:48.155943 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bb2b0-470b-44e8-b57f-3a780948cb8e None admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:48.156233 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bb2b0-470b-44e8-b57f-3a780948cb8e None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:48.156578 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bb2b0-470b-44e8-b57f-3a780948cb8e None admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:48.157331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bb2b0-470b-44e8-b57f-3a780948cb8e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:48.158859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bb2b0-470b-44e8-b57f-3a780948cb8e None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:48.204662 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 74/141] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jul 20 14:33:48 2022] POST /identity/v3/users => generated 283 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:33:48.506684 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f0655-94f5-4f0d-9003-58fb2edb6021 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:48.507023 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f0655-94f5-4f0d-9003-58fb2edb6021 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:48.507276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f0655-94f5-4f0d-9003-58fb2edb6021 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:48.508974 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 68/142] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:48.514908 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbb70f-62b5-48ae-8cb1-09ed53e35667 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:48.515239 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbb70f-62b5-48ae-8cb1-09ed53e35667 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:48.515531 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbb70f-62b5-48ae-8cb1-09ed53e35667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:48.533574 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-09cbb70f-62b5-48ae-8cb1-09ed53e35667 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:48.534831 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-09cbb70f-62b5-48ae-8cb1-09ed53e35667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:48.625954 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 75/143] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 14:33:48.660585 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f0655-94f5-4f0d-9003-58fb2edb6021 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:48.787025 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f0655-94f5-4f0d-9003-58fb2edb6021 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:48.825792 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbad587-bce9-4909-9962-d87ae53bb42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:48.826624 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbad587-bce9-4909-9962-d87ae53bb42a None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:48.826912 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbad587-bce9-4909-9962-d87ae53bb42a None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:48.827230 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbad587-bce9-4909-9962-d87ae53bb42a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:48.838463 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbad587-bce9-4909-9962-d87ae53bb42a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:33:48.846488 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-3cbad587-bce9-4909-9962-d87ae53bb42a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 14:33:48.846488 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:33:48.846488 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:33:48.846488 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:33:48.846488 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:33:48.846488 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:33:48.846488 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:33:48.846488 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:33:48.846488 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:33:48.846488 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:33:48.846488 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:33:48.846488 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:33:48.846488 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:33:48.846488 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:33:48.846488 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:33:48.846488 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:33:48.847456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:33:48.847456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:33:48.847456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:33:48.847456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:33:48.847456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:33:48.847456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:33:48.847456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:33:48.847456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:33:48.847456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:33:48.847456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:33:48.847456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:33:48.847456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:33:48.847456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:33:48.847456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:33:48.847456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:33:48.847456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:33:48.848252 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:33:48.848252 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:33:48.848252 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:33:48.848252 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:33:48.848252 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:33:48.848252 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:33:48.848252 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:33:48.848252 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:33:48.848252 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:33:48.848252 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:33:48.848252 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 14:33:48.848252 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:33:48.848252 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 69/144] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:33:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:33:48.853999 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbb70f-62b5-48ae-8cb1-09ed53e35667 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:48.915415 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:48.976040 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbb70f-62b5-48ae-8cb1-09ed53e35667 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:49.021685 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae7702a-0aed-4c2e-8dba-6ac4a3a944e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:49.022450 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7702a-0aed-4c2e-8dba-6ac4a3a944e9 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:49.023077 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7702a-0aed-4c2e-8dba-6ac4a3a944e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:49.023437 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7702a-0aed-4c2e-8dba-6ac4a3a944e9 None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:49.024260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae7702a-0aed-4c2e-8dba-6ac4a3a944e9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:49.024577 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae7702a-0aed-4c2e-8dba-6ac4a3a944e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:49.026196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae7702a-0aed-4c2e-8dba-6ac4a3a944e9 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:49.037970 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 76/145] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jul 20 14:33:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:49.042709 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbad587-bce9-4909-9962-d87ae53bb42a None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:49.046828 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:49.159961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbad587-bce9-4909-9962-d87ae53bb42a None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:49.197398 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d8340-8a0d-416a-b5b6-f6a41090dd3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:49.198317 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d8340-8a0d-416a-b5b6-f6a41090dd3f None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:49.198607 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d8340-8a0d-416a-b5b6-f6a41090dd3f None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:49.198893 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d8340-8a0d-416a-b5b6-f6a41090dd3f None admin] PATH_INFO: `/v3/users/11ef2269904344a0b9df24ec7510daf6` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:49.200229 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6d8340-8a0d-416a-b5b6-f6a41090dd3f None admin] RBAC: Authorizing `identity:get_user(user_id=11ef2269904344a0b9df24ec7510daf6)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:49.201729 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6d8340-8a0d-416a-b5b6-f6a41090dd3f None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:49.203438 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 70/146] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:33:49 2022] GET /identity/v3/users/11ef2269904344a0b9df24ec7510daf6 => generated 289 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:49.209209 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae7702a-0aed-4c2e-8dba-6ac4a3a944e9 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:49.235172 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b58b3-1c1b-4a63-a648-edc69bfe0094 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:49.235490 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b58b3-1c1b-4a63-a648-edc69bfe0094 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:49.235760 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b58b3-1c1b-4a63-a648-edc69bfe0094 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:49.237159 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 71/147] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 14:33:49.242311 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479aba9-671e-4dc0-b8af-17913feaf47c None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:49.242603 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479aba9-671e-4dc0-b8af-17913feaf47c None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:49.242887 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479aba9-671e-4dc0-b8af-17913feaf47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:49.262522 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-5479aba9-671e-4dc0-b8af-17913feaf47c None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:49.264566 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-5479aba9-671e-4dc0-b8af-17913feaf47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:49.347930 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 72/148] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 14:33:49.373033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479aba9-671e-4dc0-b8af-17913feaf47c None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:49.376806 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:49.426149 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae7702a-0aed-4c2e-8dba-6ac4a3a944e9 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:49.466780 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:49.494530 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479aba9-671e-4dc0-b8af-17913feaf47c None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:49.510966 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72fb31-8580-4167-b855-6fc70b2c8846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:49.511655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72fb31-8580-4167-b855-6fc70b2c8846 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:49.511940 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72fb31-8580-4167-b855-6fc70b2c8846 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:49.512216 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72fb31-8580-4167-b855-6fc70b2c8846 None admin] PATH_INFO: `/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:49.513479 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd72fb31-8580-4167-b855-6fc70b2c8846 None admin] RBAC: Authorizing `identity:get_project(project_id=e27e1edc913f4b99a1b0be6ab9d16b6f)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:49.515277 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd72fb31-8580-4167-b855-6fc70b2c8846 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:49.521592 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 77/149] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:33:49 2022] GET /identity/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f => generated 298 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:49.528113 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72fb31-8580-4167-b855-6fc70b2c8846 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:49.540330 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad185823-5a1a-4769-9cfe-e5521b939e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:49.541452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad185823-5a1a-4769-9cfe-e5521b939e7e None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:49.541893 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad185823-5a1a-4769-9cfe-e5521b939e7e None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:49.542319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad185823-5a1a-4769-9cfe-e5521b939e7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:49.554480 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad185823-5a1a-4769-9cfe-e5521b939e7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:33:49.563699 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-ad185823-5a1a-4769-9cfe-e5521b939e7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:33:49.563699 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:33:49.563699 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:33:49.563699 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:33:49.563699 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:33:49.563699 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:33:49.563699 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:33:49.563699 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:33:49.563699 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:33:49.563699 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:33:49.563699 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:33:49.563699 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:33:49.563699 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:33:49.563699 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:33:49.563699 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:33:49.563699 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:33:49.564607 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:33:49.564607 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:33:49.564607 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:33:49.564607 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:33:49.564607 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:33:49.564607 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:33:49.564607 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:33:49.564607 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:33:49.564607 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:33:49.564607 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:33:49.564607 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:33:49.564607 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:33:49.564607 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:33:49.564607 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:33:49.564607 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:33:49.564607 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:33:49.565377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:33:49.565377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:33:49.565377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:33:49.565377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:33:49.565377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:33:49.565377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:33:49.565377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:33:49.565377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:33:49.565377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:33:49.565377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:33:49.565377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:33:49.565377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:33:49.567571 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 73/150] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:33:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:33:49.573928 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad185823-5a1a-4769-9cfe-e5521b939e7e None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:49.677680 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72fb31-8580-4167-b855-6fc70b2c8846 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:49.714128 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:49.721963 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e094dd4-4160-4406-84e1-5a2c7c4f484d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:49.723421 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e094dd4-4160-4406-84e1-5a2c7c4f484d None admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:49.723895 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e094dd4-4160-4406-84e1-5a2c7c4f484d None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:49.724318 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e094dd4-4160-4406-84e1-5a2c7c4f484d None admin] PATH_INFO: `/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f/users/11ef2269904344a0b9df24ec7510daf6/roles/483f28cfc7444513a119d5d363f64518` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:49.726635 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e094dd4-4160-4406-84e1-5a2c7c4f484d None admin] RBAC: Authorizing `identity:create_grant(project_id=e27e1edc913f4b99a1b0be6ab9d16b6f, user_id=11ef2269904344a0b9df24ec7510daf6, role_id=483f28cfc7444513a119d5d363f64518)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:49.729392 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e094dd4-4160-4406-84e1-5a2c7c4f484d None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:49.749767 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 78/151] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jul 20 14:33:49 2022] PUT /identity/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f/users/11ef2269904344a0b9df24ec7510daf6/roles/483f28cfc7444513a119d5d363f64518 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:33:49.787292 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad185823-5a1a-4769-9cfe-e5521b939e7e None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:49.831073 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fd860-05db-43bd-9fe8-7a70aaf584a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:49.831771 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fd860-05db-43bd-9fe8-7a70aaf584a6 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:49.832057 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fd860-05db-43bd-9fe8-7a70aaf584a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:49.832334 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fd860-05db-43bd-9fe8-7a70aaf584a6 None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:49.833105 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65fd860-05db-43bd-9fe8-7a70aaf584a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:49.833413 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65fd860-05db-43bd-9fe8-7a70aaf584a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:49.835019 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65fd860-05db-43bd-9fe8-7a70aaf584a6 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:49.847539 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 74/152] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:33:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:33:49.855132 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e094dd4-4160-4406-84e1-5a2c7c4f484d None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:49.925374 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:49.998315 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e094dd4-4160-4406-84e1-5a2c7c4f484d None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:50.040564 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eff983-efed-4236-bfea-c97b6ba60094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:50.042384 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eff983-efed-4236-bfea-c97b6ba60094 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:50.042868 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eff983-efed-4236-bfea-c97b6ba60094 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:50.043390 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eff983-efed-4236-bfea-c97b6ba60094 None admin] PATH_INFO: `/v3/users/651b2280231048d59d156e569aa33194` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:50.044894 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eff983-efed-4236-bfea-c97b6ba60094 None admin] RBAC: Authorizing `identity:get_user(user_id=651b2280231048d59d156e569aa33194)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:50.046602 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eff983-efed-4236-bfea-c97b6ba60094 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:50.048493 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 79/153] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:33:49 2022] GET /identity/v3/users/651b2280231048d59d156e569aa33194 => generated 247 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:50.054658 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fd860-05db-43bd-9fe8-7a70aaf584a6 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:50.164581 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fd860-05db-43bd-9fe8-7a70aaf584a6 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:50.191347 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:50.203303 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4bea4-c12f-400f-9804-4142b7bbcbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:50.203978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4bea4-c12f-400f-9804-4142b7bbcbc3 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:50.204268 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4bea4-c12f-400f-9804-4142b7bbcbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:50.204548 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4bea4-c12f-400f-9804-4142b7bbcbc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:50.277210 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c4bea4-c12f-400f-9804-4142b7bbcbc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:50.279203 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c4bea4-c12f-400f-9804-4142b7bbcbc3 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:50.382157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 75/154] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:33:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 14:33:50.387993 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eff983-efed-4236-bfea-c97b6ba60094 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:50.495277 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eff983-efed-4236-bfea-c97b6ba60094 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:50.511426 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:50.536447 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a117022-2627-4847-b8fd-e0eb786b82c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:50.537276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a117022-2627-4847-b8fd-e0eb786b82c7 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:50.537562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a117022-2627-4847-b8fd-e0eb786b82c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:50.537837 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a117022-2627-4847-b8fd-e0eb786b82c7 None admin] PATH_INFO: `/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:50.539121 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a117022-2627-4847-b8fd-e0eb786b82c7 None admin] RBAC: Authorizing `identity:get_project(project_id=e27e1edc913f4b99a1b0be6ab9d16b6f)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:50.540611 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a117022-2627-4847-b8fd-e0eb786b82c7 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:50.542494 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 80/155] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:33:50 2022] GET /identity/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:33:50.548721 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4bea4-c12f-400f-9804-4142b7bbcbc3 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:50.579633 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:50.667934 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4bea4-c12f-400f-9804-4142b7bbcbc3 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:50.704704 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5017be-5467-4985-bcaf-4284e5de858b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:50.705397 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5017be-5467-4985-bcaf-4284e5de858b None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:50.705681 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5017be-5467-4985-bcaf-4284e5de858b None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:50.705962 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5017be-5467-4985-bcaf-4284e5de858b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:50.706817 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5017be-5467-4985-bcaf-4284e5de858b None admin] RBAC: Adding query filter params (role.id=49bd5abf255048a0abaa9538e309f516, scope.project.id=e27e1edc913f4b99a1b0be6ab9d16b6f, user.id=651b2280231048d59d156e569aa33194) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:50.707162 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5017be-5467-4985-bcaf-4284e5de858b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:50.708649 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5017be-5467-4985-bcaf-4284e5de858b None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:50.715527 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 76/156] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:33:50 2022] GET /identity/v3/role_assignments?user.id=651b2280231048d59d156e569aa33194&scope.project.id=e27e1edc913f4b99a1b0be6ab9d16b6f&role.id=49bd5abf255048a0abaa9538e309f516 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:51.006436 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c699f-356d-4453-a643-252715e5e885 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:51.006818 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c699f-356d-4453-a643-252715e5e885 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:51.007005 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c699f-356d-4453-a643-252715e5e885 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:51.008751 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 81/157] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:51.013874 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba705d-ab84-4b00-bd23-f1e9c9d394ba None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:51.014171 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba705d-ab84-4b00-bd23-f1e9c9d394ba None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:51.014452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba705d-ab84-4b00-bd23-f1e9c9d394ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:51.040185 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-22ba705d-ab84-4b00-bd23-f1e9c9d394ba None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:51.041522 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-22ba705d-ab84-4b00-bd23-f1e9c9d394ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:51.129637 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 77/158] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:33:51.162602 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c699f-356d-4453-a643-252715e5e885 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:51.277363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ebec3-0c7f-4ab2-b282-e7ff4e77cc3d None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:51.277570 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ebec3-0c7f-4ab2-b282-e7ff4e77cc3d None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:51.277828 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ebec3-0c7f-4ab2-b282-e7ff4e77cc3d None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:51.279973 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 78/159] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:51.293923 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c699f-356d-4453-a643-252715e5e885 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:51.298684 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5fb5d-daf5-47c4-83fb-8fcf67df2437 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:51.299315 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5fb5d-daf5-47c4-83fb-8fcf67df2437 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:51.299621 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5fb5d-daf5-47c4-83fb-8fcf67df2437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:51.330631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-93a5fb5d-daf5-47c4-83fb-8fcf67df2437 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:51.332965 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-93a5fb5d-daf5-47c4-83fb-8fcf67df2437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:51.351717 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd8f9c2-919f-4582-ac51-ecc1f025946a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:51.352937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8f9c2-919f-4582-ac51-ecc1f025946a None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:51.353450 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8f9c2-919f-4582-ac51-ecc1f025946a None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:51.353915 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8f9c2-919f-4582-ac51-ecc1f025946a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:51.364379 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:51.378603 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd8f9c2-919f-4582-ac51-ecc1f025946a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:33:51.387657 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-0cd8f9c2-919f-4582-ac51-ecc1f025946a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 14:33:51.387657 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:33:51.387657 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:33:51.387657 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:33:51.387657 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:33:51.387657 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:33:51.387657 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:33:51.387657 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:33:51.387657 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:33:51.387657 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:33:51.387657 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:33:51.387657 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:33:51.387657 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:33:51.387657 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:33:51.387657 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:33:51.387657 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:33:51.388571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:33:51.388571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:33:51.388571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:33:51.388571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:33:51.388571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:33:51.388571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:33:51.388571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:33:51.388571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:33:51.388571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:33:51.388571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:33:51.388571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:33:51.388571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:33:51.388571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:33:51.388571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:33:51.388571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:33:51.388571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:33:51.389343 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:33:51.389343 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:33:51.389343 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:33:51.389343 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:33:51.389343 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:33:51.389343 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:33:51.389343 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:33:51.389343 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:33:51.389343 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:33:51.389343 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:33:51.389343 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 14:33:51.389343 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:33:51.391816 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 82/160] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jul 20 14:33:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:33:51.401318 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd8f9c2-919f-4582-ac51-ecc1f025946a None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:51.496415 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 79/161] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 14:33:51.501857 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:51.574853 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5fb5d-daf5-47c4-83fb-8fcf67df2437 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:51.594318 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd8f9c2-919f-4582-ac51-ecc1f025946a None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:51.677867 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f91612-9c3d-4f87-84bc-4c25fe7bb92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:51.678564 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f91612-9c3d-4f87-84bc-4c25fe7bb92c None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:51.678852 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f91612-9c3d-4f87-84bc-4c25fe7bb92c None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:51.679172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f91612-9c3d-4f87-84bc-4c25fe7bb92c None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:51.679999 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f91612-9c3d-4f87-84bc-4c25fe7bb92c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:51.680316 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f91612-9c3d-4f87-84bc-4c25fe7bb92c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:51.681767 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f91612-9c3d-4f87-84bc-4c25fe7bb92c None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:51.705765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 83/162] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:33:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:33:51.719442 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f91612-9c3d-4f87-84bc-4c25fe7bb92c None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:51.796769 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5fb5d-daf5-47c4-83fb-8fcf67df2437 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:51.850548 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c611e8-bb85-4643-a968-b5bc0fcea4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:51.851311 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c611e8-bb85-4643-a968-b5bc0fcea4cd None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:51.851632 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c611e8-bb85-4643-a968-b5bc0fcea4cd None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:51.851912 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c611e8-bb85-4643-a968-b5bc0fcea4cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:51.868344 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c611e8-bb85-4643-a968-b5bc0fcea4cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:33:51.876617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-74c611e8-bb85-4643-a968-b5bc0fcea4cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:33:51.876617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:33:51.876617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:33:51.876617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:33:51.876617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:33:51.876617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:33:51.876617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:33:51.876617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:33:51.876617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:33:51.876617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:33:51.876617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:33:51.876617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:33:51.876617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:33:51.876617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:33:51.876617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:33:51.876617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:33:51.877682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:33:51.877682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:33:51.877682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:33:51.877682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:33:51.877682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:33:51.877682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:33:51.877682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:33:51.877682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:33:51.877682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:33:51.877682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:33:51.877682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:33:51.877682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:33:51.877682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:33:51.877682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:33:51.877682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:33:51.877682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:33:51.878667 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:33:51.878667 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:33:51.878667 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:33:51.878667 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:33:51.878667 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:33:51.878667 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:33:51.878667 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:33:51.878667 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:33:51.878667 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:33:51.878667 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:33:51.878667 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:33:51.878667 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:33:51.878667 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 80/163] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:33:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:33:51.882580 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55269b1-a6ef-4c5e-a5be-0c26a97eedfb None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:51.882938 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55269b1-a6ef-4c5e-a5be-0c26a97eedfb None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:51.883244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55269b1-a6ef-4c5e-a5be-0c26a97eedfb None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:51.884586 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 81/164] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:51.894684 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55269b1-a6ef-4c5e-a5be-0c26a97eedfb None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:51.921069 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:51.936808 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f91612-9c3d-4f87-84bc-4c25fe7bb92c None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:51.947384 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:52.022167 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dbf95a-4589-4bd4-944e-0ef4aa3fe0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:52.023184 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dbf95a-4589-4bd4-944e-0ef4aa3fe0ae None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:52.023602 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dbf95a-4589-4bd4-944e-0ef4aa3fe0ae None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:52.023875 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dbf95a-4589-4bd4-944e-0ef4aa3fe0ae None admin] PATH_INFO: `/v3/users/55e7a5b2a35b46a0a3eedafbe6c53a2c` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:52.027884 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dbf95a-4589-4bd4-944e-0ef4aa3fe0ae None admin] RBAC: Authorizing `identity:get_user(user_id=55e7a5b2a35b46a0a3eedafbe6c53a2c)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:52.029396 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dbf95a-4589-4bd4-944e-0ef4aa3fe0ae None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:52.031117 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 84/165] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:33:51 2022] GET /identity/v3/users/55e7a5b2a35b46a0a3eedafbe6c53a2c => generated 275 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:33:52.032305 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c87e8-0b3a-417d-b92d-b45b54275c52 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:52.032588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c87e8-0b3a-417d-b92d-b45b54275c52 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:52.032861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c87e8-0b3a-417d-b92d-b45b54275c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:52.079936 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55269b1-a6ef-4c5e-a5be-0c26a97eedfb None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:52.084291 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-790c87e8-0b3a-417d-b92d-b45b54275c52 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:52.085569 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-790c87e8-0b3a-417d-b92d-b45b54275c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:52.147413 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c9ca1-b2a0-4839-9434-87a0c8a0065a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:52.148181 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c9ca1-b2a0-4839-9434-87a0c8a0065a None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:52.148302 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c9ca1-b2a0-4839-9434-87a0c8a0065a None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:52.149132 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c9ca1-b2a0-4839-9434-87a0c8a0065a None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:52.152065 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c9ca1-b2a0-4839-9434-87a0c8a0065a None admin] RBAC: Adding query filter params (name=member) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:52.152387 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c9ca1-b2a0-4839-9434-87a0c8a0065a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:52.153847 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c9ca1-b2a0-4839-9434-87a0c8a0065a None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:52.185945 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 82/166] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jul 20 14:33:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:52.190433 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c9ca1-b2a0-4839-9434-87a0c8a0065a None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:52.250221 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 85/167] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:33:52.251146 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c87e8-0b3a-417d-b92d-b45b54275c52 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:52.380945 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:52.441971 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:52.445661 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c87e8-0b3a-417d-b92d-b45b54275c52 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:52.478999 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c9ca1-b2a0-4839-9434-87a0c8a0065a None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:52.532834 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c35cc0-a14f-40ff-82c3-2914000e1712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:52.533920 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c35cc0-a14f-40ff-82c3-2914000e1712 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:52.534210 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c35cc0-a14f-40ff-82c3-2914000e1712 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:52.534541 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c35cc0-a14f-40ff-82c3-2914000e1712 None admin] PATH_INFO: `/v3/users/55e7a5b2a35b46a0a3eedafbe6c53a2c` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:52.535855 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c35cc0-a14f-40ff-82c3-2914000e1712 None admin] RBAC: Authorizing `identity:get_user(user_id=55e7a5b2a35b46a0a3eedafbe6c53a2c)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:52.537380 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c35cc0-a14f-40ff-82c3-2914000e1712 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:52.539123 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 86/168] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:33:52 2022] GET /identity/v3/users/55e7a5b2a35b46a0a3eedafbe6c53a2c => generated 275 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:52.539907 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c35cc0-a14f-40ff-82c3-2914000e1712 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:52.576360 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a711ff-8b09-48b6-b387-b49d7613a390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:52.577335 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a711ff-8b09-48b6-b387-b49d7613a390 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:52.577758 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a711ff-8b09-48b6-b387-b49d7613a390 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:52.578176 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a711ff-8b09-48b6-b387-b49d7613a390 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:52.671178 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a711ff-8b09-48b6-b387-b49d7613a390 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:52.673100 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a711ff-8b09-48b6-b387-b49d7613a390 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:52.736815 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c35cc0-a14f-40ff-82c3-2914000e1712 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:52.749370 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:52.784384 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310c38ea-6280-4d0b-922d-d45843df0c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:52.785205 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c38ea-6280-4d0b-922d-d45843df0c80 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:52.785489 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c38ea-6280-4d0b-922d-d45843df0c80 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:52.785765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c38ea-6280-4d0b-922d-d45843df0c80 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:52.800087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 83/169] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:33:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:33:52.801474 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d49cb-99a1-4090-bb0c-516d359d3602 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:52.801770 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d49cb-99a1-4090-bb0c-516d359d3602 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:52.802056 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d49cb-99a1-4090-bb0c-516d359d3602 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:52.807165 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 84/170] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:52.808148 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d49cb-99a1-4090-bb0c-516d359d3602 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:52.821951 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310c38ea-6280-4d0b-922d-d45843df0c80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:33:52.835422 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-310c38ea-6280-4d0b-922d-d45843df0c80 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:33:52.835422 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:33:52.835422 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:33:52.835422 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:33:52.835422 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:33:52.835422 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:33:52.835422 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:33:52.835422 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:33:52.835422 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:33:52.835422 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:33:52.835422 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:33:52.835422 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:33:52.835422 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:33:52.835422 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:33:52.835422 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:33:52.835422 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:33:52.836581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:33:52.836581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:33:52.836581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:33:52.836581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:33:52.836581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:33:52.836581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:33:52.836581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:33:52.836581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:33:52.836581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:33:52.836581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:33:52.836581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:33:52.836581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:33:52.836581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:33:52.836581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:33:52.836581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:33:52.836581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:33:52.837536 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:33:52.837536 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:33:52.837536 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:33:52.837536 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:33:52.837536 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:33:52.837536 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:33:52.837536 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:33:52.837536 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:33:52.837536 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:33:52.837536 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:33:52.837536 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:33:52.837536 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:33:52.840531 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 87/171] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:33:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:33:52.847330 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310c38ea-6280-4d0b-922d-d45843df0c80 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:52.887107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:52.966473 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310c38ea-6280-4d0b-922d-d45843df0c80 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:53.008781 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d49cb-99a1-4090-bb0c-516d359d3602 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:53.012928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd4adef-46a4-4a0e-92e0-e95f005e4321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:53.013841 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4adef-46a4-4a0e-92e0-e95f005e4321 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:53.014127 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4adef-46a4-4a0e-92e0-e95f005e4321 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:53.014405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4adef-46a4-4a0e-92e0-e95f005e4321 None admin] PATH_INFO: `/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:53.020796 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd4adef-46a4-4a0e-92e0-e95f005e4321 None admin] RBAC: Authorizing `identity:get_project(project_id=e27e1edc913f4b99a1b0be6ab9d16b6f)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:53.022313 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd4adef-46a4-4a0e-92e0-e95f005e4321 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:53.026589 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 88/172] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:33:52 2022] GET /identity/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:53.032020 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a24a0-ba60-430b-8eb0-e4b9b9b0a73b None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:53.032315 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a24a0-ba60-430b-8eb0-e4b9b9b0a73b None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:53.032622 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a24a0-ba60-430b-8eb0-e4b9b9b0a73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:53.064099 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-8e2a24a0-ba60-430b-8eb0-e4b9b9b0a73b None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:53.068886 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-8e2a24a0-ba60-430b-8eb0-e4b9b9b0a73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:53.089691 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c786d-463c-46c8-9f9c-45d5a545b531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:53.090747 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c786d-463c-46c8-9f9c-45d5a545b531 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:53.091283 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c786d-463c-46c8-9f9c-45d5a545b531 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:53.091760 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c786d-463c-46c8-9f9c-45d5a545b531 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:53.179416 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:53.215184 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 89/173] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:33:53.216208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a24a0-ba60-430b-8eb0-e4b9b9b0a73b None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:53.243548 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5c786d-463c-46c8-9f9c-45d5a545b531 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:53.251536 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5c786d-463c-46c8-9f9c-45d5a545b531 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:53.267160 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:53.343626 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a24a0-ba60-430b-8eb0-e4b9b9b0a73b None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:53.370573 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 85/174] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:33:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:33:53.371839 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c786d-463c-46c8-9f9c-45d5a545b531 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:53.383585 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6214d1-ce93-45d8-94d9-6992415d9a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:53.384241 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6214d1-ce93-45d8-94d9-6992415d9a46 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:53.384559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6214d1-ce93-45d8-94d9-6992415d9a46 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:53.384814 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6214d1-ce93-45d8-94d9-6992415d9a46 None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:53.385597 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6214d1-ce93-45d8-94d9-6992415d9a46 None admin] RBAC: Adding query filter params (name=member) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:53.385906 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6214d1-ce93-45d8-94d9-6992415d9a46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:53.387363 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6214d1-ce93-45d8-94d9-6992415d9a46 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:53.396242 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 90/175] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jul 20 14:33:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:53.397389 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20499a-eb69-44ff-81bc-631cbd589f32 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:53.397675 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20499a-eb69-44ff-81bc-631cbd589f32 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:53.397919 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20499a-eb69-44ff-81bc-631cbd589f32 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:53.403357 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 91/176] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:53.404296 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20499a-eb69-44ff-81bc-631cbd589f32 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:53.448667 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:53.485098 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c786d-463c-46c8-9f9c-45d5a545b531 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:53.503211 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:53.516929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20499a-eb69-44ff-81bc-631cbd589f32 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:53.519118 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc46bb6c-99e2-4051-9a72-5007d7af3c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:53.519750 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46bb6c-99e2-4051-9a72-5007d7af3c38 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:53.520034 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46bb6c-99e2-4051-9a72-5007d7af3c38 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:53.520310 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46bb6c-99e2-4051-9a72-5007d7af3c38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:53.521165 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc46bb6c-99e2-4051-9a72-5007d7af3c38 None admin] RBAC: Adding query filter params (role.id=b74189174ce24c1382712858dc35f3ce, scope.project.id=e27e1edc913f4b99a1b0be6ab9d16b6f, user.id=55e7a5b2a35b46a0a3eedafbe6c53a2c) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:53.521475 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc46bb6c-99e2-4051-9a72-5007d7af3c38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:53.522965 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc46bb6c-99e2-4051-9a72-5007d7af3c38 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:53.529101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 86/177] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:33:53 2022] GET /identity/v3/role_assignments?user.id=55e7a5b2a35b46a0a3eedafbe6c53a2c&scope.project.id=e27e1edc913f4b99a1b0be6ab9d16b6f&role.id=b74189174ce24c1382712858dc35f3ce => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:53.530688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c0f80-58d3-40fa-b20d-ec3d1c099b22 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:53.530987 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c0f80-58d3-40fa-b20d-ec3d1c099b22 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:53.531297 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c0f80-58d3-40fa-b20d-ec3d1c099b22 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:53.532860 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 87/178] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:53.533589 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c0f80-58d3-40fa-b20d-ec3d1c099b22 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:53.567939 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df5ce9-071c-45b1-bafc-af6f8528e0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:53.568876 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df5ce9-071c-45b1-bafc-af6f8528e0af None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:53.569160 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df5ce9-071c-45b1-bafc-af6f8528e0af None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:53.569444 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df5ce9-071c-45b1-bafc-af6f8528e0af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:53.582367 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df5ce9-071c-45b1-bafc-af6f8528e0af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:33:53.591035 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-b2df5ce9-071c-45b1-bafc-af6f8528e0af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:33:53.591035 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:33:53.591035 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:33:53.591035 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:33:53.591035 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:33:53.591035 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:33:53.591035 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:33:53.591035 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:33:53.591035 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:33:53.591035 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:33:53.591035 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:33:53.591035 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:33:53.591035 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:33:53.591035 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:33:53.591035 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:33:53.591035 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:33:53.592036 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:33:53.592036 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:33:53.592036 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:33:53.592036 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:33:53.592036 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:33:53.592036 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:33:53.592036 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:33:53.592036 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:33:53.592036 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:33:53.592036 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:33:53.592036 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:33:53.592036 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:33:53.592036 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:33:53.592036 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:33:53.592036 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:33:53.592036 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:33:53.592811 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:33:53.592811 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:33:53.592811 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:33:53.592811 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:33:53.592811 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:33:53.592811 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:33:53.592811 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:33:53.592811 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:33:53.592811 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:33:53.592811 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:33:53.592811 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:33:53.592811 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:33:53.592811 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 92/179] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:33:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:33:53.593464 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df5ce9-071c-45b1-bafc-af6f8528e0af None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:53.658203 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c0f80-58d3-40fa-b20d-ec3d1c099b22 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:53.698642 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5316df41-7df5-4535-aa90-c2cace381836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:53.700520 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316df41-7df5-4535-aa90-c2cace381836 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:53.700520 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316df41-7df5-4535-aa90-c2cace381836 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:53.700520 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316df41-7df5-4535-aa90-c2cace381836 None admin] PATH_INFO: `/v3/projects/21b6e12113924026b1b402ed414f7518` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:53.701288 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5316df41-7df5-4535-aa90-c2cace381836 None admin] RBAC: Authorizing `identity:get_project(project_id=21b6e12113924026b1b402ed414f7518)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:53.702795 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5316df41-7df5-4535-aa90-c2cace381836 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:53.705112 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 88/180] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:33:53 2022] GET /identity/v3/projects/21b6e12113924026b1b402ed414f7518 => generated 312 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:53.705964 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac1f92-78c9-4a7e-b416-8e48d56ae6a0 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:53.706248 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac1f92-78c9-4a7e-b416-8e48d56ae6a0 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:53.706494 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac1f92-78c9-4a7e-b416-8e48d56ae6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:53.711590 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:53.717313 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df5ce9-071c-45b1-bafc-af6f8528e0af None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:53.724762 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-e9ac1f92-78c9-4a7e-b416-8e48d56ae6a0 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:53.728847 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e9ac1f92-78c9-4a7e-b416-8e48d56ae6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:53.757908 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d548f-20cd-4cc4-96ac-84f17ea24988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:53.758915 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d548f-20cd-4cc4-96ac-84f17ea24988 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:53.759370 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d548f-20cd-4cc4-96ac-84f17ea24988 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:53.760237 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d548f-20cd-4cc4-96ac-84f17ea24988 None admin] PATH_INFO: `/v3/users/55e7a5b2a35b46a0a3eedafbe6c53a2c` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:53.761466 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82d548f-20cd-4cc4-96ac-84f17ea24988 None admin] RBAC: Authorizing `identity:get_user(user_id=55e7a5b2a35b46a0a3eedafbe6c53a2c)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:53.763202 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82d548f-20cd-4cc4-96ac-84f17ea24988 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:53.764641 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 93/181] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:33:53 2022] GET /identity/v3/users/55e7a5b2a35b46a0a3eedafbe6c53a2c => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:53.765863 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ac44d-9dfa-4659-ac08-12923553cb39 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:53.766146 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ac44d-9dfa-4659-ac08-12923553cb39 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:53.766423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ac44d-9dfa-4659-ac08-12923553cb39 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:53.771157 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 94/182] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:53.771157 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7105d9-c62c-42d7-9bcb-f7261af8914e None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:53.771157 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7105d9-c62c-42d7-9bcb-f7261af8914e None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:53.771157 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7105d9-c62c-42d7-9bcb-f7261af8914e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:53.786700 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-cf7105d9-c62c-42d7-9bcb-f7261af8914e None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:53.787910 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-cf7105d9-c62c-42d7-9bcb-f7261af8914e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:53.799362 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:53.810049 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 89/183] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:33:53.810712 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac1f92-78c9-4a7e-b416-8e48d56ae6a0 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:53.859075 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 95/184] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:33:53.859977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7105d9-c62c-42d7-9bcb-f7261af8914e None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:53.917928 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac1f92-78c9-4a7e-b416-8e48d56ae6a0 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:53.949742 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:53.956581 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03318e2-1fc3-47ef-9fea-694f9684a57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:53.957278 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03318e2-1fc3-47ef-9fea-694f9684a57e None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:53.957575 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03318e2-1fc3-47ef-9fea-694f9684a57e None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:53.957870 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03318e2-1fc3-47ef-9fea-694f9684a57e None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:53.958887 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03318e2-1fc3-47ef-9fea-694f9684a57e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:53.958999 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03318e2-1fc3-47ef-9fea-694f9684a57e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:53.960451 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03318e2-1fc3-47ef-9fea-694f9684a57e None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:53.971583 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 90/185] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:33:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:53.972218 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03318e2-1fc3-47ef-9fea-694f9684a57e None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:53.976817 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7105d9-c62c-42d7-9bcb-f7261af8914e None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:54.011073 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6b794-c9cb-4874-a2b8-043a07813bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:54.011771 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6b794-c9cb-4874-a2b8-043a07813bf1 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:54.012059 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6b794-c9cb-4874-a2b8-043a07813bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:54.012338 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6b794-c9cb-4874-a2b8-043a07813bf1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:54.013176 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db6b794-c9cb-4874-a2b8-043a07813bf1 None admin] RBAC: Adding query filter params (role.id=7e394dd183b147ffa15b617f076bbb76, scope.project.id=21b6e12113924026b1b402ed414f7518, user.id=55e7a5b2a35b46a0a3eedafbe6c53a2c) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:54.013491 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db6b794-c9cb-4874-a2b8-043a07813bf1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:54.014958 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db6b794-c9cb-4874-a2b8-043a07813bf1 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:54.020706 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 96/186] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:33:53 2022] GET /identity/v3/role_assignments?user.id=55e7a5b2a35b46a0a3eedafbe6c53a2c&scope.project.id=21b6e12113924026b1b402ed414f7518&role.id=7e394dd183b147ffa15b617f076bbb76 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:54.022022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d94be3-7010-48c2-a817-c63dec8ead1e None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:54.022249 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d94be3-7010-48c2-a817-c63dec8ead1e None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:54.022861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d94be3-7010-48c2-a817-c63dec8ead1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:54.040949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-33d94be3-7010-48c2-a817-c63dec8ead1e None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:54.042225 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-33d94be3-7010-48c2-a817-c63dec8ead1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:54.045818 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:54.088946 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03318e2-1fc3-47ef-9fea-694f9684a57e None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:54.124938 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 97/187] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:33:54.126163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d94be3-7010-48c2-a817-c63dec8ead1e None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:54.137515 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56fa05-ad2a-46c0-b23c-5538c5d3cb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:54.138207 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56fa05-ad2a-46c0-b23c-5538c5d3cb1b None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:54.138488 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56fa05-ad2a-46c0-b23c-5538c5d3cb1b None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:54.138760 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56fa05-ad2a-46c0-b23c-5538c5d3cb1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:54.206562 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:54.217645 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56fa05-ad2a-46c0-b23c-5538c5d3cb1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:54.219240 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56fa05-ad2a-46c0-b23c-5538c5d3cb1b None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:54.254879 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d94be3-7010-48c2-a817-c63dec8ead1e None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:54.291639 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9647d0-68a8-46e0-8755-08559f179e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:54.292515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9647d0-68a8-46e0-8755-08559f179e89 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:54.292808 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9647d0-68a8-46e0-8755-08559f179e89 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:54.293092 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9647d0-68a8-46e0-8755-08559f179e89 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:54.302258 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:54.305860 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9647d0-68a8-46e0-8755-08559f179e89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:33:54.317815 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-bb9647d0-68a8-46e0-8755-08559f179e89 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 14:33:54.317815 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:33:54.317815 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:33:54.317815 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:33:54.317815 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:33:54.317815 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:33:54.317815 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:33:54.317815 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:33:54.317815 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:33:54.317815 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:33:54.317815 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:33:54.317815 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:33:54.317815 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:33:54.317815 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:33:54.317815 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:33:54.317815 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:33:54.318797 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:33:54.318797 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:33:54.318797 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:33:54.318797 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:33:54.318797 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:33:54.318797 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:33:54.318797 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:33:54.318797 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:33:54.318797 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:33:54.318797 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:33:54.318797 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:33:54.318797 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:33:54.318797 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:33:54.318797 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:33:54.318797 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:33:54.318797 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:33:54.319653 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:33:54.319653 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:33:54.319653 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:33:54.319653 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:33:54.319653 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:33:54.319653 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:33:54.319653 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:33:54.319653 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:33:54.319653 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:33:54.319653 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:33:54.319653 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 14:33:54.319653 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:33:54.319653 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 98/188] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jul 20 14:33:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:33:54.322895 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9647d0-68a8-46e0-8755-08559f179e89 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:54.337126 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 91/189] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:33:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:33:54.337987 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56fa05-ad2a-46c0-b23c-5538c5d3cb1b None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:54.429358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9647d0-68a8-46e0-8755-08559f179e89 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:54.430474 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:54.437502 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56fa05-ad2a-46c0-b23c-5538c5d3cb1b None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:54.462612 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e76700d-d770-45e1-9767-6745a976ff3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:54.463491 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76700d-d770-45e1-9767-6745a976ff3f None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:54.463762 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76700d-d770-45e1-9767-6745a976ff3f None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:54.464024 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76700d-d770-45e1-9767-6745a976ff3f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:54.472193 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a421c3-20b1-447a-a8d2-475da6e7c675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:54.473104 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a421c3-20b1-447a-a8d2-475da6e7c675 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:54.473372 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a421c3-20b1-447a-a8d2-475da6e7c675 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:54.473656 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a421c3-20b1-447a-a8d2-475da6e7c675 None admin] PATH_INFO: `/v3/users/8b6466e9d2284acfb0a0d945a895c9bd` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:54.474143 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e76700d-d770-45e1-9767-6745a976ff3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:33:54.481949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-2e76700d-d770-45e1-9767-6745a976ff3f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 14:33:54.481949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:33:54.481949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:33:54.481949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:33:54.481949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:33:54.481949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:33:54.481949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:33:54.481949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:33:54.481949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:33:54.481949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:33:54.481949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:33:54.481949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:33:54.481949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:33:54.481949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:33:54.481949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:33:54.481949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:33:54.483013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:33:54.483013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:33:54.483013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:33:54.483013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:33:54.483013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:33:54.483013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:33:54.483013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:33:54.483013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:33:54.483013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:33:54.483013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:33:54.483013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:33:54.483013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:33:54.483013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:33:54.483013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:33:54.483013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:33:54.483013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:33:54.483861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:33:54.483861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:33:54.483861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:33:54.483861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:33:54.483861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:33:54.483861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:33:54.483861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:33:54.483861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:33:54.483861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:33:54.483861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:33:54.483861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 14:33:54.483861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:33:54.483861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 99/190] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:33:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:33:54.487947 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e76700d-d770-45e1-9767-6745a976ff3f None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:54.494086 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a421c3-20b1-447a-a8d2-475da6e7c675 None admin] RBAC: Authorizing `identity:get_user(user_id=8b6466e9d2284acfb0a0d945a895c9bd)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:54.495682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a421c3-20b1-447a-a8d2-475da6e7c675 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:54.497870 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 92/191] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:33:54 2022] GET /identity/v3/users/8b6466e9d2284acfb0a0d945a895c9bd => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:54.498716 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a421c3-20b1-447a-a8d2-475da6e7c675 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:54.547816 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:54.589256 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e76700d-d770-45e1-9767-6745a976ff3f None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:54.597748 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a421c3-20b1-447a-a8d2-475da6e7c675 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:54.622880 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7dcde1-9e3a-45ca-98a5-7b17392f052a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:54.623722 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7dcde1-9e3a-45ca-98a5-7b17392f052a None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:54.623977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7dcde1-9e3a-45ca-98a5-7b17392f052a None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:54.624277 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7dcde1-9e3a-45ca-98a5-7b17392f052a None admin] PATH_INFO: `/v3/domains/default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:54.625506 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7dcde1-9e3a-45ca-98a5-7b17392f052a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:54.626971 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7dcde1-9e3a-45ca-98a5-7b17392f052a None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:54.629109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 100/192] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:33:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:54.629383 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7dcde1-9e3a-45ca-98a5-7b17392f052a None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:54.634806 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02044ba6-ce1e-4bb6-bc89-b2cac46bb207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:54.635650 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02044ba6-ce1e-4bb6-bc89-b2cac46bb207 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:54.635938 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02044ba6-ce1e-4bb6-bc89-b2cac46bb207 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:54.636217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02044ba6-ce1e-4bb6-bc89-b2cac46bb207 None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:54.637010 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02044ba6-ce1e-4bb6-bc89-b2cac46bb207 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:54.637323 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02044ba6-ce1e-4bb6-bc89-b2cac46bb207 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:54.638767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02044ba6-ce1e-4bb6-bc89-b2cac46bb207 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:54.648126 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 93/193] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:33:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:54.648904 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02044ba6-ce1e-4bb6-bc89-b2cac46bb207 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:54.669271 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:54.731650 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7dcde1-9e3a-45ca-98a5-7b17392f052a None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:54.753915 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02044ba6-ce1e-4bb6-bc89-b2cac46bb207 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:54.757701 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:54.766332 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba3a80b-9055-4e22-852f-7c0fc7266aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:54.767248 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba3a80b-9055-4e22-852f-7c0fc7266aeb None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:54.767533 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba3a80b-9055-4e22-852f-7c0fc7266aeb None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:54.767812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba3a80b-9055-4e22-852f-7c0fc7266aeb None admin] PATH_INFO: `/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:54.769018 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba3a80b-9055-4e22-852f-7c0fc7266aeb None admin] RBAC: Authorizing `identity:get_project(project_id=e27e1edc913f4b99a1b0be6ab9d16b6f)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:54.770493 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba3a80b-9055-4e22-852f-7c0fc7266aeb None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:54.772250 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 101/194] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:33:54 2022] GET /identity/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:54.776160 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba3a80b-9055-4e22-852f-7c0fc7266aeb None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:54.787816 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550494f-315b-402a-842b-34ecf387c6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:54.788507 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550494f-315b-402a-842b-34ecf387c6e2 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:54.788787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550494f-315b-402a-842b-34ecf387c6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:54.789063 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550494f-315b-402a-842b-34ecf387c6e2 None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:54.789821 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f550494f-315b-402a-842b-34ecf387c6e2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:54.790128 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f550494f-315b-402a-842b-34ecf387c6e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:54.791572 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f550494f-315b-402a-842b-34ecf387c6e2 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:54.800481 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 94/195] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jul 20 14:33:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:54.801261 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550494f-315b-402a-842b-34ecf387c6e2 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:54.882521 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:54.886327 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba3a80b-9055-4e22-852f-7c0fc7266aeb None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:54.906425 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550494f-315b-402a-842b-34ecf387c6e2 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:54.919992 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bff062-c6e1-44b6-b23c-72250be49e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:54.920690 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bff062-c6e1-44b6-b23c-72250be49e09 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:54.920986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bff062-c6e1-44b6-b23c-72250be49e09 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:54.921249 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bff062-c6e1-44b6-b23c-72250be49e09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:54.941168 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565372f8-ca90-4bbf-a82e-a2f1c541940c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:54.941856 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565372f8-ca90-4bbf-a82e-a2f1c541940c None admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:54.942136 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565372f8-ca90-4bbf-a82e-a2f1c541940c None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:54.942421 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565372f8-ca90-4bbf-a82e-a2f1c541940c None admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:54.943314 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565372f8-ca90-4bbf-a82e-a2f1c541940c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:54.944779 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565372f8-ca90-4bbf-a82e-a2f1c541940c None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:54.966168 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 95/196] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jul 20 14:33:54 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:33:54.968029 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565372f8-ca90-4bbf-a82e-a2f1c541940c None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:54.992347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:55.004978 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bff062-c6e1-44b6-b23c-72250be49e09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:55.006495 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bff062-c6e1-44b6-b23c-72250be49e09 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:55.087611 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565372f8-ca90-4bbf-a82e-a2f1c541940c None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:55.121426 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:55.123824 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 102/197] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:33:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:33:55.124890 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bff062-c6e1-44b6-b23c-72250be49e09 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:55.127374 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dbd74f-aaaa-4bee-b294-14cd79c28ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:55.129313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbd74f-aaaa-4bee-b294-14cd79c28ffd None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:55.129807 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbd74f-aaaa-4bee-b294-14cd79c28ffd None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:55.131165 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbd74f-aaaa-4bee-b294-14cd79c28ffd None admin] PATH_INFO: `/v3/users/8b6466e9d2284acfb0a0d945a895c9bd` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:55.133091 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dbd74f-aaaa-4bee-b294-14cd79c28ffd None admin] RBAC: Authorizing `identity:get_user(user_id=8b6466e9d2284acfb0a0d945a895c9bd)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:55.135022 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dbd74f-aaaa-4bee-b294-14cd79c28ffd None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:55.137049 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 96/198] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:33:54 2022] GET /identity/v3/users/8b6466e9d2284acfb0a0d945a895c9bd => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:55.137690 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dbd74f-aaaa-4bee-b294-14cd79c28ffd None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:55.240068 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:55.247893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bff062-c6e1-44b6-b23c-72250be49e09 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:55.265687 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dbd74f-aaaa-4bee-b294-14cd79c28ffd None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:55.292092 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7e35a-ac4d-40d7-bb7d-3bcd239eec1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:55.292791 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7e35a-ac4d-40d7-bb7d-3bcd239eec1e None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:55.293080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7e35a-ac4d-40d7-bb7d-3bcd239eec1e None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:55.293360 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7e35a-ac4d-40d7-bb7d-3bcd239eec1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:55.294595 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7e35a-ac4d-40d7-bb7d-3bcd239eec1e None admin] RBAC: Adding query filter params (role.id=7e394dd183b147ffa15b617f076bbb76, scope.project.id=e27e1edc913f4b99a1b0be6ab9d16b6f, user.id=55e7a5b2a35b46a0a3eedafbe6c53a2c) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:55.294912 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7e35a-ac4d-40d7-bb7d-3bcd239eec1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:55.296457 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7e35a-ac4d-40d7-bb7d-3bcd239eec1e None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:55.304662 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 103/199] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:33:55 2022] GET /identity/v3/role_assignments?user.id=55e7a5b2a35b46a0a3eedafbe6c53a2c&scope.project.id=e27e1edc913f4b99a1b0be6ab9d16b6f&role.id=7e394dd183b147ffa15b617f076bbb76 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:55.305456 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7e35a-ac4d-40d7-bb7d-3bcd239eec1e None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:55.308657 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b0612-60e3-4d36-a672-955d3ac51aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:55.311111 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b0612-60e3-4d36-a672-955d3ac51aaf None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:55.311626 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b0612-60e3-4d36-a672-955d3ac51aaf None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:55.312067 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b0612-60e3-4d36-a672-955d3ac51aaf None admin] PATH_INFO: `/v3/users/11ef2269904344a0b9df24ec7510daf6` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:55.313757 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b0612-60e3-4d36-a672-955d3ac51aaf None admin] RBAC: Authorizing `identity:get_user(user_id=11ef2269904344a0b9df24ec7510daf6)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:55.315529 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b0612-60e3-4d36-a672-955d3ac51aaf None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:55.318326 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 97/200] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:33:55 2022] GET /identity/v3/users/11ef2269904344a0b9df24ec7510daf6 => generated 289 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:55.319448 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b0612-60e3-4d36-a672-955d3ac51aaf None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:55.412005 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:55.450726 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b0612-60e3-4d36-a672-955d3ac51aaf None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:55.452655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7e35a-ac4d-40d7-bb7d-3bcd239eec1e None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:55.517355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607275b3-b66c-4c3e-bf1b-b37e86f1f384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:55.518820 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607275b3-b66c-4c3e-bf1b-b37e86f1f384 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:55.520593 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607275b3-b66c-4c3e-bf1b-b37e86f1f384 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:55.521178 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607275b3-b66c-4c3e-bf1b-b37e86f1f384 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:55.521993 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da1a4e5-2ef9-4127-8a13-f1bf1906beec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:55.523809 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1a4e5-2ef9-4127-8a13-f1bf1906beec None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:55.526375 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1a4e5-2ef9-4127-8a13-f1bf1906beec None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:55.526880 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1a4e5-2ef9-4127-8a13-f1bf1906beec None admin] PATH_INFO: `/v3/projects/888e6ef050b643db90e7280d61f3e3e8` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:55.528630 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da1a4e5-2ef9-4127-8a13-f1bf1906beec None admin] RBAC: Authorizing `identity:get_project(project_id=888e6ef050b643db90e7280d61f3e3e8)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:55.531149 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da1a4e5-2ef9-4127-8a13-f1bf1906beec None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:55.537993 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 104/201] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:33:55 2022] GET /identity/v3/projects/888e6ef050b643db90e7280d61f3e3e8 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:55.543143 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da1a4e5-2ef9-4127-8a13-f1bf1906beec None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:55.578301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:55.612608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607275b3-b66c-4c3e-bf1b-b37e86f1f384 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:55.614113 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607275b3-b66c-4c3e-bf1b-b37e86f1f384 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:55.666547 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da1a4e5-2ef9-4127-8a13-f1bf1906beec None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:55.708681 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:55.724610 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b17e62-2a9c-44a9-b213-f8e7eb0982ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:55.726149 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b17e62-2a9c-44a9-b213-f8e7eb0982ba None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:55.726648 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b17e62-2a9c-44a9-b213-f8e7eb0982ba None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:55.727149 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b17e62-2a9c-44a9-b213-f8e7eb0982ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:55.775420 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 98/202] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:33:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:33:55.779822 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607275b3-b66c-4c3e-bf1b-b37e86f1f384 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:55.810030 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b17e62-2a9c-44a9-b213-f8e7eb0982ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:55.812278 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b17e62-2a9c-44a9-b213-f8e7eb0982ba None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:55.855436 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:55.906470 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607275b3-b66c-4c3e-bf1b-b37e86f1f384 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:55.945023 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a0c3b8-d07e-483b-8071-24fa15513c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:55.945880 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0c3b8-d07e-483b-8071-24fa15513c54 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:55.946294 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0c3b8-d07e-483b-8071-24fa15513c54 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:55.946668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0c3b8-d07e-483b-8071-24fa15513c54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:55.947659 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a0c3b8-d07e-483b-8071-24fa15513c54 None admin] RBAC: Adding query filter params (role.id=49bd5abf255048a0abaa9538e309f516, scope.project.id=888e6ef050b643db90e7280d61f3e3e8, user.id=8b6466e9d2284acfb0a0d945a895c9bd) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:55.948072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a0c3b8-d07e-483b-8071-24fa15513c54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:55.949401 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 105/203] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:33:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:33:55.950409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b17e62-2a9c-44a9-b213-f8e7eb0982ba None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:55.954274 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a0c3b8-d07e-483b-8071-24fa15513c54 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:55.960791 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 99/204] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:33:55 2022] GET /identity/v3/role_assignments?user.id=8b6466e9d2284acfb0a0d945a895c9bd&scope.project.id=888e6ef050b643db90e7280d61f3e3e8&role.id=49bd5abf255048a0abaa9538e309f516 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:55.967085 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a0c3b8-d07e-483b-8071-24fa15513c54 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:56.035881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:56.192187 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b17e62-2a9c-44a9-b213-f8e7eb0982ba None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:56.195912 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a0c3b8-d07e-483b-8071-24fa15513c54 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:56.240993 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2293531-2246-43e5-8d79-ab03c7715d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:56.241895 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2293531-2246-43e5-8d79-ab03c7715d80 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:56.242251 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2293531-2246-43e5-8d79-ab03c7715d80 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:56.242534 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2293531-2246-43e5-8d79-ab03c7715d80 None admin] PATH_INFO: `/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:56.247591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2293531-2246-43e5-8d79-ab03c7715d80 None admin] RBAC: Authorizing `identity:get_project(project_id=e27e1edc913f4b99a1b0be6ab9d16b6f)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:56.249077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2293531-2246-43e5-8d79-ab03c7715d80 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:56.250864 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 100/205] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:33:55 2022] GET /identity/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f => generated 298 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:56.257373 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2293531-2246-43e5-8d79-ab03c7715d80 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:56.262339 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:56.281618 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a9d9a-568c-4127-ae8b-3dbf57c33a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:56.282879 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a9d9a-568c-4127-ae8b-3dbf57c33a19 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:56.283386 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a9d9a-568c-4127-ae8b-3dbf57c33a19 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:56.283874 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a9d9a-568c-4127-ae8b-3dbf57c33a19 None admin] PATH_INFO: `/v3/projects/888e6ef050b643db90e7280d61f3e3e8` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:56.285312 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a9d9a-568c-4127-ae8b-3dbf57c33a19 None admin] RBAC: Authorizing `identity:get_project(project_id=888e6ef050b643db90e7280d61f3e3e8)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:56.287002 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a9d9a-568c-4127-ae8b-3dbf57c33a19 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:56.289121 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 106/206] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:33:55 2022] GET /identity/v3/projects/888e6ef050b643db90e7280d61f3e3e8 => generated 302 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:56.307378 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a9d9a-568c-4127-ae8b-3dbf57c33a19 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:56.394933 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:56.398757 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2293531-2246-43e5-8d79-ab03c7715d80 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:56.454727 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409774ce-f48e-42a4-95a1-a705a67967f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:56.455418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409774ce-f48e-42a4-95a1-a705a67967f0 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:56.455796 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409774ce-f48e-42a4-95a1-a705a67967f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:56.456079 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409774ce-f48e-42a4-95a1-a705a67967f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:56.457061 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409774ce-f48e-42a4-95a1-a705a67967f0 None admin] RBAC: Adding query filter params (role.id=483f28cfc7444513a119d5d363f64518, scope.project.id=e27e1edc913f4b99a1b0be6ab9d16b6f, user.id=11ef2269904344a0b9df24ec7510daf6) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:56.458543 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409774ce-f48e-42a4-95a1-a705a67967f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:56.462597 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409774ce-f48e-42a4-95a1-a705a67967f0 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:56.471081 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 101/207] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:33:56 2022] GET /identity/v3/role_assignments?user.id=11ef2269904344a0b9df24ec7510daf6&scope.project.id=e27e1edc913f4b99a1b0be6ab9d16b6f&role.id=483f28cfc7444513a119d5d363f64518 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:56.495684 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a9d9a-568c-4127-ae8b-3dbf57c33a19 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:56.536244 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524fdc71-c757-44c9-9237-40a8fd6e8544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:56.536925 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fdc71-c757-44c9-9237-40a8fd6e8544 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:56.537201 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fdc71-c757-44c9-9237-40a8fd6e8544 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:56.537477 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fdc71-c757-44c9-9237-40a8fd6e8544 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:56.538324 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524fdc71-c757-44c9-9237-40a8fd6e8544 None admin] RBAC: Adding query filter params (role.id=b74189174ce24c1382712858dc35f3ce, scope.project.id=888e6ef050b643db90e7280d61f3e3e8, user.id=8b6466e9d2284acfb0a0d945a895c9bd) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:56.538695 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524fdc71-c757-44c9-9237-40a8fd6e8544 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:56.540238 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524fdc71-c757-44c9-9237-40a8fd6e8544 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:56.546639 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 107/208] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:33:56 2022] GET /identity/v3/role_assignments?user.id=8b6466e9d2284acfb0a0d945a895c9bd&scope.project.id=888e6ef050b643db90e7280d61f3e3e8&role.id=b74189174ce24c1382712858dc35f3ce => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:33:57.370579 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce63d6-3fd8-4ff1-a8bb-a29abf3bbd48 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:57.371451 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce63d6-3fd8-4ff1-a8bb-a29abf3bbd48 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:57.371949 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce63d6-3fd8-4ff1-a8bb-a29abf3bbd48 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:57.373801 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 102/209] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:57.379465 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82964aba-6cc1-4c4a-9251-99204e1f60ca None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:57.380296 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82964aba-6cc1-4c4a-9251-99204e1f60ca None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:57.380772 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82964aba-6cc1-4c4a-9251-99204e1f60ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:57.400256 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-82964aba-6cc1-4c4a-9251-99204e1f60ca None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:57.401809 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-82964aba-6cc1-4c4a-9251-99204e1f60ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:57.420994 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8f7fc-8cd7-4651-a6e5-81a98ffe371b None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:57.421516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8f7fc-8cd7-4651-a6e5-81a98ffe371b None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:57.421516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8f7fc-8cd7-4651-a6e5-81a98ffe371b None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:57.423102 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 103/210] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:57.428550 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8bd7b-a099-48c2-9268-8edbada41ebe None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:57.430010 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8bd7b-a099-48c2-9268-8edbada41ebe None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:57.430010 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8bd7b-a099-48c2-9268-8edbada41ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:57.449669 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-e0c8bd7b-a099-48c2-9268-8edbada41ebe None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:57.450864 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e0c8bd7b-a099-48c2-9268-8edbada41ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:57.479415 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:57.484240 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 108/211] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:33:57.509650 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82964aba-6cc1-4c4a-9251-99204e1f60ca None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:57.536689 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 104/212] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 14:33:57.561550 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8bd7b-a099-48c2-9268-8edbada41ebe None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:57.626423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82964aba-6cc1-4c4a-9251-99204e1f60ca None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:57.667149 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d0115f-9ddf-4ba0-aa32-03664b16d7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:57.667970 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0115f-9ddf-4ba0-aa32-03664b16d7a8 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:57.668252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0115f-9ddf-4ba0-aa32-03664b16d7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:57.668528 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0115f-9ddf-4ba0-aa32-03664b16d7a8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:57.679546 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:57.682775 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d0115f-9ddf-4ba0-aa32-03664b16d7a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:33:57.688353 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8bd7b-a099-48c2-9268-8edbada41ebe None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:57.695574 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-89d0115f-9ddf-4ba0-aa32-03664b16d7a8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:33:57.695574 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:33:57.695574 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:33:57.695574 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:33:57.695574 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:33:57.695574 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:33:57.695574 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:33:57.695574 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:33:57.695574 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:33:57.695574 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:33:57.695574 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:33:57.695574 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:33:57.695574 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:33:57.695574 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:33:57.695574 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:33:57.695574 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:33:57.696482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:33:57.696482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:33:57.696482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:33:57.696482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:33:57.696482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:33:57.696482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:33:57.696482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:33:57.696482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:33:57.696482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:33:57.696482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:33:57.696482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:33:57.696482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:33:57.696482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:33:57.696482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:33:57.696482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:33:57.696482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:33:57.697272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:33:57.697272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:33:57.697272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:33:57.697272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:33:57.697272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:33:57.697272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:33:57.697272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:33:57.697272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:33:57.697272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:33:57.697272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:33:57.697272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:33:57.697272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:33:57.697272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 109/213] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:33:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:33:57.702622 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d0115f-9ddf-4ba0-aa32-03664b16d7a8 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:57.732799 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d45b7-cc80-42d5-a245-32a50b5c4a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:57.734619 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d45b7-cc80-42d5-a245-32a50b5c4a73 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:57.735423 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d45b7-cc80-42d5-a245-32a50b5c4a73 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:57.735937 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d45b7-cc80-42d5-a245-32a50b5c4a73 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:57.738088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d45b7-cc80-42d5-a245-32a50b5c4a73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:57.740545 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d45b7-cc80-42d5-a245-32a50b5c4a73 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:57.747405 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 105/214] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:33:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:33:57.749666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d45b7-cc80-42d5-a245-32a50b5c4a73 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:57.770395 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:57.827999 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d0115f-9ddf-4ba0-aa32-03664b16d7a8 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:57.858300 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d45b7-cc80-42d5-a245-32a50b5c4a73 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:57.865419 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9103491-9b62-46e0-89ec-be9e9f313e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:57.866483 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9103491-9b62-46e0-89ec-be9e9f313e37 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:57.866955 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9103491-9b62-46e0-89ec-be9e9f313e37 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:57.867569 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9103491-9b62-46e0-89ec-be9e9f313e37 None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:57.868611 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9103491-9b62-46e0-89ec-be9e9f313e37 None admin] RBAC: Adding query filter params (name=member) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:57.869120 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9103491-9b62-46e0-89ec-be9e9f313e37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:57.870885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9103491-9b62-46e0-89ec-be9e9f313e37 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:57.881415 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 110/215] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jul 20 14:33:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:33:57.887107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9103491-9b62-46e0-89ec-be9e9f313e37 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:57.894303 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c96a5d9-f6c0-401a-bf56-51f705ea1d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:57.895007 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96a5d9-f6c0-401a-bf56-51f705ea1d35 None admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:57.895322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96a5d9-f6c0-401a-bf56-51f705ea1d35 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:57.895603 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96a5d9-f6c0-401a-bf56-51f705ea1d35 None admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:57.896344 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c96a5d9-f6c0-401a-bf56-51f705ea1d35 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:57.897815 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c96a5d9-f6c0-401a-bf56-51f705ea1d35 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:57.923661 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 106/216] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jul 20 14:33:57 2022] POST /identity/v3/users => generated 297 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:33:57.997832 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9103491-9b62-46e0-89ec-be9e9f313e37 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:58.013307 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:58.038815 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481da656-ad5d-497d-bd00-34598f650fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:58.040156 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481da656-ad5d-497d-bd00-34598f650fd9 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:58.040582 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481da656-ad5d-497d-bd00-34598f650fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:58.041044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481da656-ad5d-497d-bd00-34598f650fd9 None admin] PATH_INFO: `/v3/users/68c8c7a1c47147c6b4956416370e3e2a` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:58.060817 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481da656-ad5d-497d-bd00-34598f650fd9 None admin] RBAC: Authorizing `identity:get_user(user_id=68c8c7a1c47147c6b4956416370e3e2a)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:58.062350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481da656-ad5d-497d-bd00-34598f650fd9 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:58.064539 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 111/217] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:33:57 2022] GET /identity/v3/users/68c8c7a1c47147c6b4956416370e3e2a => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:58.070183 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c96a5d9-f6c0-401a-bf56-51f705ea1d35 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:58.097270 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:58.188110 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c96a5d9-f6c0-401a-bf56-51f705ea1d35 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:58.224057 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad9a2e-d120-4a52-b737-738f78897558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:58.224727 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad9a2e-d120-4a52-b737-738f78897558 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:58.225032 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad9a2e-d120-4a52-b737-738f78897558 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:58.225285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad9a2e-d120-4a52-b737-738f78897558 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:58.273874 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ed0c7-a532-4b84-ae87-f9f619cea789 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:58.274182 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ed0c7-a532-4b84-ae87-f9f619cea789 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:58.274427 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ed0c7-a532-4b84-ae87-f9f619cea789 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:58.275965 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 112/218] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:58.283190 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74cdee-70fb-472b-9b84-b6e91dd08d44 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:58.283480 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74cdee-70fb-472b-9b84-b6e91dd08d44 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:58.283760 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74cdee-70fb-472b-9b84-b6e91dd08d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:58.306083 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bad9a2e-d120-4a52-b737-738f78897558 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:58.307638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bad9a2e-d120-4a52-b737-738f78897558 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:58.310425 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-1a74cdee-70fb-472b-9b84-b6e91dd08d44 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:58.311780 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-1a74cdee-70fb-472b-9b84-b6e91dd08d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:58.330159 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:58.401998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 113/219] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:33:58.426052 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a74cdee-70fb-472b-9b84-b6e91dd08d44 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:58.436507 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 107/220] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:33:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:33:58.444853 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad9a2e-d120-4a52-b737-738f78897558 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:58.603075 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad9a2e-d120-4a52-b737-738f78897558 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:58.608176 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:58.614034 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a74cdee-70fb-472b-9b84-b6e91dd08d44 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:58.619369 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:58.638806 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d2494-43b5-4c6d-ac4a-b1cbe948c264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:58.639786 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d2494-43b5-4c6d-ac4a-b1cbe948c264 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:58.640069 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d2494-43b5-4c6d-ac4a-b1cbe948c264 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:58.640349 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d2494-43b5-4c6d-ac4a-b1cbe948c264 None admin] PATH_INFO: `/v3/projects/888e6ef050b643db90e7280d61f3e3e8` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:58.641735 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9d2494-43b5-4c6d-ac4a-b1cbe948c264 None admin] RBAC: Authorizing `identity:get_project(project_id=888e6ef050b643db90e7280d61f3e3e8)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:58.643413 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9d2494-43b5-4c6d-ac4a-b1cbe948c264 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:58.645435 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 108/221] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:33:58 2022] GET /identity/v3/projects/888e6ef050b643db90e7280d61f3e3e8 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:33:58.653394 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d2494-43b5-4c6d-ac4a-b1cbe948c264 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:58.708018 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e085ea9e-7ca2-441f-917f-eb0d49ecadb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:58.708773 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e085ea9e-7ca2-441f-917f-eb0d49ecadb3 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:58.709067 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e085ea9e-7ca2-441f-917f-eb0d49ecadb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:58.709344 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e085ea9e-7ca2-441f-917f-eb0d49ecadb3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:58.739985 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e085ea9e-7ca2-441f-917f-eb0d49ecadb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:33:58.748137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-e085ea9e-7ca2-441f-917f-eb0d49ecadb3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:33:58.748137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:33:58.748137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:33:58.748137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:33:58.748137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:33:58.748137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:33:58.748137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:33:58.748137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:33:58.748137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:33:58.748137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:33:58.748137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:33:58.748137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:33:58.748137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:33:58.748137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:33:58.748137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:33:58.748137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:33:58.749039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:33:58.749039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:33:58.749039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:33:58.749039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:33:58.749039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:33:58.749039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:33:58.749039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:33:58.749039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:33:58.749039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:33:58.749039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:33:58.749039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:33:58.749039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:33:58.749039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:33:58.749039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:33:58.749039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:33:58.749039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:33:58.749846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:33:58.749846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:33:58.749846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:33:58.749846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:33:58.749846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:33:58.749846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:33:58.749846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:33:58.749846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:33:58.749846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:33:58.749846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:33:58.749846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:33:58.749846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:33:58.749846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 114/222] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:33:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:33:58.755498 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e085ea9e-7ca2-441f-917f-eb0d49ecadb3 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:58.765756 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d2494-43b5-4c6d-ac4a-b1cbe948c264 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:58.802198 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30edc0d-7801-4b68-984e-6aec2b83d865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:58.802909 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30edc0d-7801-4b68-984e-6aec2b83d865 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:58.803249 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30edc0d-7801-4b68-984e-6aec2b83d865 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:58.803526 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30edc0d-7801-4b68-984e-6aec2b83d865 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:58.804375 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30edc0d-7801-4b68-984e-6aec2b83d865 None admin] RBAC: Adding query filter params (role.id=7e394dd183b147ffa15b617f076bbb76, scope.project.id=888e6ef050b643db90e7280d61f3e3e8, user.id=68c8c7a1c47147c6b4956416370e3e2a) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:58.804834 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30edc0d-7801-4b68-984e-6aec2b83d865 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:58.806301 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30edc0d-7801-4b68-984e-6aec2b83d865 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:58.812746 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 109/223] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:33:58 2022] GET /identity/v3/role_assignments?user.id=68c8c7a1c47147c6b4956416370e3e2a&scope.project.id=888e6ef050b643db90e7280d61f3e3e8&role.id=7e394dd183b147ffa15b617f076bbb76 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:33:58.814104 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06abbe1-f7f4-4370-8fe3-a2a6ec5248e6 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:58.814388 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06abbe1-f7f4-4370-8fe3-a2a6ec5248e6 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:58.814665 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06abbe1-f7f4-4370-8fe3-a2a6ec5248e6 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:58.824389 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 110/224] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:33:58 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:33:58.832630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba702d30-e92a-44aa-a572-7c000d9aaa72 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:58.832964 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba702d30-e92a-44aa-a572-7c000d9aaa72 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:58.833190 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba702d30-e92a-44aa-a572-7c000d9aaa72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:58.855978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-ba702d30-e92a-44aa-a572-7c000d9aaa72 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:33:58.863081 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-ba702d30-e92a-44aa-a572-7c000d9aaa72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:33:58.912281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:58.929728 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e085ea9e-7ca2-441f-917f-eb0d49ecadb3 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:58.945815 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 111/225] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:33:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:33:58.971210 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba702d30-e92a-44aa-a572-7c000d9aaa72 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:58.996261 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591734af-634e-4ffc-aedb-55ade72b85da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:58.997359 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591734af-634e-4ffc-aedb-55ade72b85da None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:58.997864 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591734af-634e-4ffc-aedb-55ade72b85da None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:58.998337 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591734af-634e-4ffc-aedb-55ade72b85da None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:58.999367 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591734af-634e-4ffc-aedb-55ade72b85da None admin] RBAC: Adding query filter params (name=admin) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:58.999935 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591734af-634e-4ffc-aedb-55ade72b85da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:59.001549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591734af-634e-4ffc-aedb-55ade72b85da None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:59.019762 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 115/226] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:33:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:59.026680 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591734af-634e-4ffc-aedb-55ade72b85da None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:59.080949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:59.092104 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba702d30-e92a-44aa-a572-7c000d9aaa72 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:59.135871 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba953c2a-652f-4396-9206-649a209b53bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:59.136719 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba953c2a-652f-4396-9206-649a209b53bf None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:59.137000 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba953c2a-652f-4396-9206-649a209b53bf None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:59.137280 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba953c2a-652f-4396-9206-649a209b53bf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:59.157870 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba953c2a-652f-4396-9206-649a209b53bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:33:59.170270 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-ba953c2a-652f-4396-9206-649a209b53bf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 14:33:59.170270 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:33:59.170270 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:33:59.170270 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:33:59.170270 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:33:59.170270 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:33:59.170270 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:33:59.170270 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:33:59.170270 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:33:59.170270 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:33:59.170270 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:33:59.170270 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:33:59.170270 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:33:59.170270 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:33:59.170270 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:33:59.170270 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:33:59.171264 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:33:59.171264 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:33:59.171264 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:33:59.171264 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:33:59.171264 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:33:59.171264 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:33:59.171264 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:33:59.171264 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:33:59.171264 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:33:59.171264 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:33:59.171264 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:33:59.171264 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:33:59.171264 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:33:59.171264 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:33:59.171264 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:33:59.171264 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:33:59.172075 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:33:59.172075 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:33:59.172075 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:33:59.172075 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:33:59.172075 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:33:59.172075 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:33:59.172075 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:33:59.172075 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:33:59.172075 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:33:59.172075 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:33:59.172075 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 14:33:59.172075 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:33:59.173148 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 112/227] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jul 20 14:33:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:33:59.181738 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba953c2a-652f-4396-9206-649a209b53bf None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:59.234436 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591734af-634e-4ffc-aedb-55ade72b85da None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:59.243071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:59.287286 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f081e10-47a3-456e-9b78-3611f18cb52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:59.288288 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f081e10-47a3-456e-9b78-3611f18cb52c None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:59.288573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f081e10-47a3-456e-9b78-3611f18cb52c None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:59.288887 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f081e10-47a3-456e-9b78-3611f18cb52c None admin] PATH_INFO: `/v3/users/8b6466e9d2284acfb0a0d945a895c9bd` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:59.290111 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f081e10-47a3-456e-9b78-3611f18cb52c None admin] RBAC: Authorizing `identity:get_user(user_id=8b6466e9d2284acfb0a0d945a895c9bd)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:59.291898 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f081e10-47a3-456e-9b78-3611f18cb52c None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:59.293743 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 116/228] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:33:59 2022] GET /identity/v3/users/8b6466e9d2284acfb0a0d945a895c9bd => generated 283 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:59.305576 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba953c2a-652f-4396-9206-649a209b53bf None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:59.308771 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f081e10-47a3-456e-9b78-3611f18cb52c None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:59.368665 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbf65a-bbeb-4601-943c-abed529089d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:59.369362 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbf65a-bbeb-4601-943c-abed529089d4 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:59.369650 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbf65a-bbeb-4601-943c-abed529089d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:59.369934 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbf65a-bbeb-4601-943c-abed529089d4 None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:59.370718 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebbf65a-bbeb-4601-943c-abed529089d4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:33:59.371068 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebbf65a-bbeb-4601-943c-abed529089d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:59.372523 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebbf65a-bbeb-4601-943c-abed529089d4 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:59.387508 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 113/229] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:33:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:59.395082 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbf65a-bbeb-4601-943c-abed529089d4 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:59.463964 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:59.477273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f081e10-47a3-456e-9b78-3611f18cb52c None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:59.543733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45a9f6-b91a-4cd8-90b0-4db4fd896f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:59.545655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45a9f6-b91a-4cd8-90b0-4db4fd896f22 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:59.548483 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45a9f6-b91a-4cd8-90b0-4db4fd896f22 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:59.549193 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45a9f6-b91a-4cd8-90b0-4db4fd896f22 None admin] PATH_INFO: `/v3/projects/888e6ef050b643db90e7280d61f3e3e8` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:59.551548 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad45a9f6-b91a-4cd8-90b0-4db4fd896f22 None admin] RBAC: Authorizing `identity:get_project(project_id=888e6ef050b643db90e7280d61f3e3e8)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:59.553045 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad45a9f6-b91a-4cd8-90b0-4db4fd896f22 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:59.555124 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 117/230] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:33:59 2022] GET /identity/v3/projects/888e6ef050b643db90e7280d61f3e3e8 => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:33:59.575873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45a9f6-b91a-4cd8-90b0-4db4fd896f22 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:59.632098 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbf65a-bbeb-4601-943c-abed529089d4 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:59.636023 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:33:59.702375 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80378b7-d6b7-4e4c-8e0c-15645d36c92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:59.703312 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80378b7-d6b7-4e4c-8e0c-15645d36c92e None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:59.703655 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80378b7-d6b7-4e4c-8e0c-15645d36c92e None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:59.703935 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80378b7-d6b7-4e4c-8e0c-15645d36c92e None admin] PATH_INFO: `/v3/users/8b6466e9d2284acfb0a0d945a895c9bd` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:59.705898 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80378b7-d6b7-4e4c-8e0c-15645d36c92e None admin] RBAC: Authorizing `identity:get_user(user_id=8b6466e9d2284acfb0a0d945a895c9bd)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:59.707409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80378b7-d6b7-4e4c-8e0c-15645d36c92e None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:59.715576 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 114/231] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:33:59 2022] GET /identity/v3/users/8b6466e9d2284acfb0a0d945a895c9bd => generated 283 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:33:59.724104 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80378b7-d6b7-4e4c-8e0c-15645d36c92e None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:59.773916 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45a9f6-b91a-4cd8-90b0-4db4fd896f22 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:59.844067 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7369c0a-2e00-4569-8e66-22b6b5c36ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:59.845019 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7369c0a-2e00-4569-8e66-22b6b5c36ade None admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:59.845297 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7369c0a-2e00-4569-8e66-22b6b5c36ade None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:59.845572 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7369c0a-2e00-4569-8e66-22b6b5c36ade None admin] PATH_INFO: `/v3/projects/888e6ef050b643db90e7280d61f3e3e8/users/8b6466e9d2284acfb0a0d945a895c9bd/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:59.852609 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7369c0a-2e00-4569-8e66-22b6b5c36ade None admin] RBAC: Authorizing `identity:create_grant(project_id=888e6ef050b643db90e7280d61f3e3e8, user_id=8b6466e9d2284acfb0a0d945a895c9bd, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:59.854220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7369c0a-2e00-4569-8e66-22b6b5c36ade None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:59.885599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 118/232] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jul 20 14:33:59 2022] PUT /identity/v3/projects/888e6ef050b643db90e7280d61f3e3e8/users/8b6466e9d2284acfb0a0d945a895c9bd/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 14:33:59.886871 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80378b7-d6b7-4e4c-8e0c-15645d36c92e None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:33:59.925388 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad124647-13cf-4b92-9ebf-1de1a364e564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:33:59.926295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad124647-13cf-4b92-9ebf-1de1a364e564 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:33:59.926577 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad124647-13cf-4b92-9ebf-1de1a364e564 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:33:59.926857 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad124647-13cf-4b92-9ebf-1de1a364e564 None admin] PATH_INFO: `/v3/projects/888e6ef050b643db90e7280d61f3e3e8` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:33:59.928145 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad124647-13cf-4b92-9ebf-1de1a364e564 None admin] RBAC: Authorizing `identity:get_project(project_id=888e6ef050b643db90e7280d61f3e3e8)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:33:59.929715 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad124647-13cf-4b92-9ebf-1de1a364e564 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:33:59.931855 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 115/233] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:33:59 2022] GET /identity/v3/projects/888e6ef050b643db90e7280d61f3e3e8 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:33:59.937778 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7369c0a-2e00-4569-8e66-22b6b5c36ade None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:33:59.987498 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:00.100414 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7369c0a-2e00-4569-8e66-22b6b5c36ade None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:00.137785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8094c-4931-43e6-bfe3-bff6a8edf5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:00.138703 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8094c-4931-43e6-bfe3-bff6a8edf5bb None admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:00.138979 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8094c-4931-43e6-bfe3-bff6a8edf5bb None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:00.139278 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8094c-4931-43e6-bfe3-bff6a8edf5bb None admin] PATH_INFO: `/v3/projects/888e6ef050b643db90e7280d61f3e3e8/users/8b6466e9d2284acfb0a0d945a895c9bd/roles/b74189174ce24c1382712858dc35f3ce` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:00.141529 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d8094c-4931-43e6-bfe3-bff6a8edf5bb None admin] RBAC: Authorizing `identity:create_grant(project_id=888e6ef050b643db90e7280d61f3e3e8, user_id=8b6466e9d2284acfb0a0d945a895c9bd, role_id=b74189174ce24c1382712858dc35f3ce)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:00.143009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d8094c-4931-43e6-bfe3-bff6a8edf5bb None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:00.158176 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 119/234] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jul 20 14:33:59 2022] PUT /identity/v3/projects/888e6ef050b643db90e7280d61f3e3e8/users/8b6466e9d2284acfb0a0d945a895c9bd/roles/b74189174ce24c1382712858dc35f3ce => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:34:00.631654 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f51764-c25d-4bc5-8a5e-3dfc6efa3795 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:00.632134 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f51764-c25d-4bc5-8a5e-3dfc6efa3795 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:00.632716 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f51764-c25d-4bc5-8a5e-3dfc6efa3795 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:00.635305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 116/235] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:00.641253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de59fd1f-e17d-46f4-b9bd-d9aafffa9e22 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:00.641730 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de59fd1f-e17d-46f4-b9bd-d9aafffa9e22 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:00.641830 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de59fd1f-e17d-46f4-b9bd-d9aafffa9e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:00.668798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-de59fd1f-e17d-46f4-b9bd-d9aafffa9e22 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:00.670764 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-de59fd1f-e17d-46f4-b9bd-d9aafffa9e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:00.772901 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea5e45-23ce-4b1a-8569-2f700d70cb4a None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:00.773280 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea5e45-23ce-4b1a-8569-2f700d70cb4a None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:00.773455 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea5e45-23ce-4b1a-8569-2f700d70cb4a None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:00.774843 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 117/236] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:00.782073 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 120/237] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:00.787634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e40b0-ee4b-4157-b7ff-409748a74f65 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:00.787929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e40b0-ee4b-4157-b7ff-409748a74f65 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:00.788209 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e40b0-ee4b-4157-b7ff-409748a74f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:00.806568 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de59fd1f-e17d-46f4-b9bd-d9aafffa9e22 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:00.807927 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-c70e40b0-ee4b-4157-b7ff-409748a74f65 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:00.809173 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-c70e40b0-ee4b-4157-b7ff-409748a74f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:00.856271 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:00.875109 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:00.960165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de59fd1f-e17d-46f4-b9bd-d9aafffa9e22 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:00.975692 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 118/238] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:01.019098 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367293c-e54d-47cf-8735-907fbdf832a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:01.019944 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367293c-e54d-47cf-8735-907fbdf832a1 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:01.020225 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367293c-e54d-47cf-8735-907fbdf832a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:01.020496 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367293c-e54d-47cf-8735-907fbdf832a1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:01.036336 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1367293c-e54d-47cf-8735-907fbdf832a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:01.045214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-1367293c-e54d-47cf-8735-907fbdf832a1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 14:34:01.045214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:01.045214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:01.045214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:01.045214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:01.045214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:01.045214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:01.045214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:01.045214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:01.045214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:01.045214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:01.045214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:01.045214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:01.045214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:01.045214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:01.045214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:01.046150 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:01.046150 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:01.046150 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:01.046150 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:01.046150 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:01.046150 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:01.046150 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:01.046150 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:01.046150 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:01.046150 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:01.046150 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:01.046150 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:01.046150 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:01.046150 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:01.046150 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:01.046150 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:01.047463 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:01.047463 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:01.047463 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:01.047463 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:01.047463 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:01.047463 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:01.047463 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:01.047463 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:01.047463 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:01.047463 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:01.047463 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 14:34:01.047463 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:01.047463 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 121/239] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:34:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:34:01.052714 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e40b0-ee4b-4157-b7ff-409748a74f65 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:01.052917 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367293c-e54d-47cf-8735-907fbdf832a1 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:01.147009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:01.210027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367293c-e54d-47cf-8735-907fbdf832a1 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:01.242081 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e40b0-ee4b-4157-b7ff-409748a74f65 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:01.261795 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621bff0a-66f7-4f79-8bae-35bce24833c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:01.263466 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621bff0a-66f7-4f79-8bae-35bce24833c4 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:01.263952 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621bff0a-66f7-4f79-8bae-35bce24833c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:01.264547 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621bff0a-66f7-4f79-8bae-35bce24833c4 None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:01.265505 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621bff0a-66f7-4f79-8bae-35bce24833c4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:01.266014 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621bff0a-66f7-4f79-8bae-35bce24833c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:01.268231 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621bff0a-66f7-4f79-8bae-35bce24833c4 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:01.282054 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d877e0-bf33-450b-95c3-2b0e98786ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:01.283286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d877e0-bf33-450b-95c3-2b0e98786ca1 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:01.283904 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d877e0-bf33-450b-95c3-2b0e98786ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:01.284420 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d877e0-bf33-450b-95c3-2b0e98786ca1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:01.286002 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d877e0-bf33-450b-95c3-2b0e98786ca1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:01.286481 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 122/240] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jul 20 14:34:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:34:01.287941 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab6c89-3c66-4606-92ee-576adba3fbd3 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:01.288236 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab6c89-3c66-4606-92ee-576adba3fbd3 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:01.288498 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d877e0-bf33-450b-95c3-2b0e98786ca1 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:01.288614 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab6c89-3c66-4606-92ee-576adba3fbd3 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:01.290181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 123/241] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:01.294862 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab6c89-3c66-4606-92ee-576adba3fbd3 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:01.295376 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 119/242] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:01.297118 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab887779-7f3c-4aab-afd3-5bd71fd036b6 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:01.297628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab887779-7f3c-4aab-afd3-5bd71fd036b6 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:01.298103 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab887779-7f3c-4aab-afd3-5bd71fd036b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:01.316566 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-ab887779-7f3c-4aab-afd3-5bd71fd036b6 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:01.318157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-ab887779-7f3c-4aab-afd3-5bd71fd036b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:01.351035 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:01.396721 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 120/243] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:01.404319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab887779-7f3c-4aab-afd3-5bd71fd036b6 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:01.432418 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab6c89-3c66-4606-92ee-576adba3fbd3 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:01.471404 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec895c63-bcad-487c-af6c-511e541fdb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:01.472355 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec895c63-bcad-487c-af6c-511e541fdb8d None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:01.472645 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec895c63-bcad-487c-af6c-511e541fdb8d None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:01.472924 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec895c63-bcad-487c-af6c-511e541fdb8d None admin] PATH_INFO: `/v3/users/717529633938498bb7dd18cfeff4989f` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:01.475505 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:01.500094 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec895c63-bcad-487c-af6c-511e541fdb8d None admin] RBAC: Authorizing `identity:get_user(user_id=717529633938498bb7dd18cfeff4989f)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:01.502023 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec895c63-bcad-487c-af6c-511e541fdb8d None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:01.504025 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 124/244] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:34:01 2022] GET /identity/v3/users/717529633938498bb7dd18cfeff4989f => generated 297 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:01.504970 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec895c63-bcad-487c-af6c-511e541fdb8d None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:01.558673 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab887779-7f3c-4aab-afd3-5bd71fd036b6 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:01.607660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9366ac0-0e16-48c1-8476-cc4ba52facd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:01.608765 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9366ac0-0e16-48c1-8476-cc4ba52facd8 None admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:01.609052 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9366ac0-0e16-48c1-8476-cc4ba52facd8 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:01.609329 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9366ac0-0e16-48c1-8476-cc4ba52facd8 None admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:01.610065 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9366ac0-0e16-48c1-8476-cc4ba52facd8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:01.612214 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9366ac0-0e16-48c1-8476-cc4ba52facd8 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:01.623874 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec895c63-bcad-487c-af6c-511e541fdb8d None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:01.643153 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 121/245] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jul 20 14:34:01 2022] POST /identity/v3/users => generated 293 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:34:01.644275 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9366ac0-0e16-48c1-8476-cc4ba52facd8 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:01.693546 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35034a06-c5d9-409d-9144-7d291052357b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:01.694386 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35034a06-c5d9-409d-9144-7d291052357b None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:01.694670 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35034a06-c5d9-409d-9144-7d291052357b None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:01.694947 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35034a06-c5d9-409d-9144-7d291052357b None admin] PATH_INFO: `/v3/roles/member` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:01.701289 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:01.716044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35034a06-c5d9-409d-9144-7d291052357b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:01.724350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-35034a06-c5d9-409d-9144-7d291052357b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:34:01.724350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:01.724350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:01.724350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:01.724350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:01.724350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:01.724350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:01.724350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:01.724350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:01.724350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:01.724350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:01.724350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:01.724350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:01.724350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:01.724350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:01.724350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:01.725345 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:01.725345 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:01.725345 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:01.725345 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:01.725345 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:01.725345 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:01.725345 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:01.725345 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:01.725345 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:01.725345 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:01.725345 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:01.725345 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:01.725345 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:01.725345 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:01.725345 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:01.725345 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:01.726205 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:01.726205 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:01.726205 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:01.726205 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:01.726205 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:01.726205 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:01.726205 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:01.726205 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:01.726205 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:01.726205 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:01.726205 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:34:01.726205 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:01.727195 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 125/246] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:34:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:01.732401 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35034a06-c5d9-409d-9144-7d291052357b None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:01.806685 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9366ac0-0e16-48c1-8476-cc4ba52facd8 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:01.849323 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa7019-1cc0-4abe-b18c-6a328dee2bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:01.850375 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa7019-1cc0-4abe-b18c-6a328dee2bda None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:01.850884 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa7019-1cc0-4abe-b18c-6a328dee2bda None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:01.851436 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa7019-1cc0-4abe-b18c-6a328dee2bda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:01.854418 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:01.936599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35034a06-c5d9-409d-9144-7d291052357b None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:01.950635 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daa7019-1cc0-4abe-b18c-6a328dee2bda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:01.952617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daa7019-1cc0-4abe-b18c-6a328dee2bda None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:01.973489 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cfec48-37b7-42de-8f55-50457a581069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:01.974182 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cfec48-37b7-42de-8f55-50457a581069 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:01.974474 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cfec48-37b7-42de-8f55-50457a581069 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:01.974742 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cfec48-37b7-42de-8f55-50457a581069 None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:01.975535 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cfec48-37b7-42de-8f55-50457a581069 None admin] RBAC: Adding query filter params (name=member) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:01.975840 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cfec48-37b7-42de-8f55-50457a581069 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:01.977370 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cfec48-37b7-42de-8f55-50457a581069 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:01.981124 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:01.990253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 126/247] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jul 20 14:34:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:01.995516 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cfec48-37b7-42de-8f55-50457a581069 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:02.093181 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 122/248] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:34:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:34:02.101121 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa7019-1cc0-4abe-b18c-6a328dee2bda None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:02.123648 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cfec48-37b7-42de-8f55-50457a581069 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:02.143524 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:02.196100 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a6fbe-9bb7-4d59-a8c3-dd58c985d47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:02.196100 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a6fbe-9bb7-4d59-a8c3-dd58c985d47b None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:02.196100 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a6fbe-9bb7-4d59-a8c3-dd58c985d47b None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:02.196605 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a6fbe-9bb7-4d59-a8c3-dd58c985d47b None admin] PATH_INFO: `/v3/users/68c8c7a1c47147c6b4956416370e3e2a` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:02.198476 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a6fbe-9bb7-4d59-a8c3-dd58c985d47b None admin] RBAC: Authorizing `identity:get_user(user_id=68c8c7a1c47147c6b4956416370e3e2a)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:02.200058 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a6fbe-9bb7-4d59-a8c3-dd58c985d47b None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:02.201835 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 127/249] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:34:01 2022] GET /identity/v3/users/68c8c7a1c47147c6b4956416370e3e2a => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:02.209461 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a6fbe-9bb7-4d59-a8c3-dd58c985d47b None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:02.263697 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa7019-1cc0-4abe-b18c-6a328dee2bda None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:02.297700 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:02.314817 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c73c648-a0bb-4b04-a8e7-79860f741c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:02.317050 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73c648-a0bb-4b04-a8e7-79860f741c9f None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:02.317581 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73c648-a0bb-4b04-a8e7-79860f741c9f None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:02.318047 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73c648-a0bb-4b04-a8e7-79860f741c9f None admin] PATH_INFO: `/v3/projects/888e6ef050b643db90e7280d61f3e3e8` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:02.319551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c73c648-a0bb-4b04-a8e7-79860f741c9f None admin] RBAC: Authorizing `identity:get_project(project_id=888e6ef050b643db90e7280d61f3e3e8)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:02.321861 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c73c648-a0bb-4b04-a8e7-79860f741c9f None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:02.324668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 123/250] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:34:02 2022] GET /identity/v3/projects/888e6ef050b643db90e7280d61f3e3e8 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:02.332324 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c73c648-a0bb-4b04-a8e7-79860f741c9f None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:02.361749 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a6fbe-9bb7-4d59-a8c3-dd58c985d47b None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:02.407820 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19126295-5768-4934-adf9-06deb8be3266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:02.409768 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19126295-5768-4934-adf9-06deb8be3266 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:02.410294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19126295-5768-4934-adf9-06deb8be3266 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:02.410789 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19126295-5768-4934-adf9-06deb8be3266 None admin] PATH_INFO: `/v3/projects/888e6ef050b643db90e7280d61f3e3e8` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:02.412848 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19126295-5768-4934-adf9-06deb8be3266 None admin] RBAC: Authorizing `identity:get_project(project_id=888e6ef050b643db90e7280d61f3e3e8)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:02.415312 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19126295-5768-4934-adf9-06deb8be3266 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:02.418909 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 128/251] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:34:02 2022] GET /identity/v3/projects/888e6ef050b643db90e7280d61f3e3e8 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:02.426235 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19126295-5768-4934-adf9-06deb8be3266 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:02.483753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:02.492123 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c73c648-a0bb-4b04-a8e7-79860f741c9f None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:02.542956 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c3ab7-ab7b-4f23-a815-a6a98a758f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:02.544130 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c3ab7-ab7b-4f23-a815-a6a98a758f1b None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:02.544620 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c3ab7-ab7b-4f23-a815-a6a98a758f1b None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:02.545080 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c3ab7-ab7b-4f23-a815-a6a98a758f1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:02.546210 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8c3ab7-ab7b-4f23-a815-a6a98a758f1b None admin] RBAC: Adding query filter params (role.id=483f28cfc7444513a119d5d363f64518, scope.project.id=888e6ef050b643db90e7280d61f3e3e8, user.id=717529633938498bb7dd18cfeff4989f) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:02.546703 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8c3ab7-ab7b-4f23-a815-a6a98a758f1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:02.548394 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8c3ab7-ab7b-4f23-a815-a6a98a758f1b None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:02.559094 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 124/252] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:34:02 2022] GET /identity/v3/role_assignments?user.id=717529633938498bb7dd18cfeff4989f&scope.project.id=888e6ef050b643db90e7280d61f3e3e8&role.id=483f28cfc7444513a119d5d363f64518 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:02.559094 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbdd01-e25f-46b1-ad64-3907fb6a50c6 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:02.559094 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbdd01-e25f-46b1-ad64-3907fb6a50c6 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:02.559094 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbdd01-e25f-46b1-ad64-3907fb6a50c6 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:02.560184 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 125/253] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:02.566057 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0276e4-db4a-4f96-a685-abdd36f78e05 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:02.566446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0276e4-db4a-4f96-a685-abdd36f78e05 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:02.566832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0276e4-db4a-4f96-a685-abdd36f78e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:02.595090 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19126295-5768-4934-adf9-06deb8be3266 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:02.596609 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-6c0276e4-db4a-4f96-a685-abdd36f78e05 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:02.598009 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-6c0276e4-db4a-4f96-a685-abdd36f78e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:02.696466 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:02.715323 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca3459-0987-4d7c-920b-a987b3c17f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:02.715323 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca3459-0987-4d7c-920b-a987b3c17f31 None admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:02.715323 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca3459-0987-4d7c-920b-a987b3c17f31 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:02.715323 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca3459-0987-4d7c-920b-a987b3c17f31 None admin] PATH_INFO: `/v3/projects/888e6ef050b643db90e7280d61f3e3e8/users/68c8c7a1c47147c6b4956416370e3e2a/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:02.724549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ca3459-0987-4d7c-920b-a987b3c17f31 None admin] RBAC: Authorizing `identity:create_grant(project_id=888e6ef050b643db90e7280d61f3e3e8, user_id=68c8c7a1c47147c6b4956416370e3e2a, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:02.731084 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ca3459-0987-4d7c-920b-a987b3c17f31 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:02.733648 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 126/254] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 14:34:02.766362 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3a217-523c-4553-b575-2b892ac21758 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:02.767038 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3a217-523c-4553-b575-2b892ac21758 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:02.767653 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3a217-523c-4553-b575-2b892ac21758 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:02.769666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 127/255] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:02.772831 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 129/256] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jul 20 14:34:02 2022] PUT /identity/v3/projects/888e6ef050b643db90e7280d61f3e3e8/users/68c8c7a1c47147c6b4956416370e3e2a/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:34:02.776088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce147075-d203-4a23-bfbf-7a3755404aa2 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:02.776596 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce147075-d203-4a23-bfbf-7a3755404aa2 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:02.777028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce147075-d203-4a23-bfbf-7a3755404aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:02.782245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca3459-0987-4d7c-920b-a987b3c17f31 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:02.810677 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-ce147075-d203-4a23-bfbf-7a3755404aa2 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:02.812791 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-ce147075-d203-4a23-bfbf-7a3755404aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:02.944078 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 128/257] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 14:34:02.993486 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce147075-d203-4a23-bfbf-7a3755404aa2 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:03.040133 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca3459-0987-4d7c-920b-a987b3c17f31 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:03.055215 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:03.113471 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21efde52-7de8-4a42-8e7c-62f90bf0fc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:03.114388 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21efde52-7de8-4a42-8e7c-62f90bf0fc83 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:03.114672 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21efde52-7de8-4a42-8e7c-62f90bf0fc83 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:03.114983 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21efde52-7de8-4a42-8e7c-62f90bf0fc83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:03.142574 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21efde52-7de8-4a42-8e7c-62f90bf0fc83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:03.154453 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce147075-d203-4a23-bfbf-7a3755404aa2 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:03.155385 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-21efde52-7de8-4a42-8e7c-62f90bf0fc83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:34:03.155385 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:03.155385 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:03.155385 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:03.155385 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:03.155385 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:03.155385 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:03.155385 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:03.155385 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:03.155385 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:03.155385 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:03.155385 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:03.155385 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:03.155385 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:03.155385 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:03.155385 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:03.156913 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:03.156913 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:03.156913 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:03.156913 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:03.156913 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:03.156913 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:03.156913 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:03.156913 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:03.156913 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:03.156913 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:03.156913 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:03.156913 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:03.156913 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:03.156913 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:03.156913 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:03.156913 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:03.158399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:03.158399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:03.158399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:03.158399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:03.158399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:03.158399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:03.158399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:03.158399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:03.158399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:03.158399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:03.158399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:34:03.158399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:03.158399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 130/258] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:34:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:03.166224 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21efde52-7de8-4a42-8e7c-62f90bf0fc83 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:03.172868 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:03.226505 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923848d-d2ac-4217-92cb-7fecc142a6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:03.227398 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923848d-d2ac-4217-92cb-7fecc142a6dc None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:03.229106 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923848d-d2ac-4217-92cb-7fecc142a6dc None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:03.229388 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923848d-d2ac-4217-92cb-7fecc142a6dc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:03.253007 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5923848d-d2ac-4217-92cb-7fecc142a6dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:03.270432 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-5923848d-d2ac-4217-92cb-7fecc142a6dc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 14:34:03.270432 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:03.270432 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:03.270432 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:03.270432 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:03.270432 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:03.270432 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:03.270432 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:03.270432 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:03.270432 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:03.270432 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:03.270432 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:03.270432 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:03.270432 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:03.270432 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:03.270432 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:03.271549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:03.271549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:03.271549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:03.271549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:03.271549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:03.271549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:03.271549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:03.271549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:03.271549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:03.271549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:03.271549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:03.271549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:03.271549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:03.271549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:03.271549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:03.271549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:03.272352 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:03.272352 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:03.272352 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:03.272352 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:03.272352 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:03.272352 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:03.272352 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:03.272352 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:03.272352 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:03.272352 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:03.272352 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 14:34:03.272352 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:03.272961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 129/259] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jul 20 14:34:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:03.284634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923848d-d2ac-4217-92cb-7fecc142a6dc None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:03.374141 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21efde52-7de8-4a42-8e7c-62f90bf0fc83 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:03.413298 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea75c4fa-57f5-4780-b950-7321c1e7f9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:03.414228 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75c4fa-57f5-4780-b950-7321c1e7f9c4 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:03.414515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75c4fa-57f5-4780-b950-7321c1e7f9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:03.414796 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75c4fa-57f5-4780-b950-7321c1e7f9c4 None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:03.415609 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea75c4fa-57f5-4780-b950-7321c1e7f9c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:03.415923 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea75c4fa-57f5-4780-b950-7321c1e7f9c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:03.417691 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea75c4fa-57f5-4780-b950-7321c1e7f9c4 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:03.428057 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 131/260] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:34:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:34:03.436820 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea75c4fa-57f5-4780-b950-7321c1e7f9c4 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:03.493823 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:03.611755 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea75c4fa-57f5-4780-b950-7321c1e7f9c4 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:03.633214 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923848d-d2ac-4217-92cb-7fecc142a6dc None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:03.677605 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc44ce8-8931-42ea-a32e-5342b225b37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:03.678871 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc44ce8-8931-42ea-a32e-5342b225b37b None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:03.679326 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc44ce8-8931-42ea-a32e-5342b225b37b None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:03.679770 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc44ce8-8931-42ea-a32e-5342b225b37b None admin] PATH_INFO: `/v3/users/8b6466e9d2284acfb0a0d945a895c9bd` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:03.682913 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc44ce8-8931-42ea-a32e-5342b225b37b None admin] RBAC: Authorizing `identity:get_user(user_id=8b6466e9d2284acfb0a0d945a895c9bd)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:03.684634 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc44ce8-8931-42ea-a32e-5342b225b37b None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:03.686781 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 132/261] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:34:03 2022] GET /identity/v3/users/8b6466e9d2284acfb0a0d945a895c9bd => generated 283 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:03.692761 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc44ce8-8931-42ea-a32e-5342b225b37b None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:03.723827 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b8268e-4c6a-420b-bdc3-46282e84258c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:03.724682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8268e-4c6a-420b-bdc3-46282e84258c None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:03.725086 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8268e-4c6a-420b-bdc3-46282e84258c None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:03.725506 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8268e-4c6a-420b-bdc3-46282e84258c None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:03.726576 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b8268e-4c6a-420b-bdc3-46282e84258c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:03.726966 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b8268e-4c6a-420b-bdc3-46282e84258c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:03.729931 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b8268e-4c6a-420b-bdc3-46282e84258c None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:03.745903 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 130/262] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:34:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 462 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:03.755185 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b8268e-4c6a-420b-bdc3-46282e84258c None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:03.836262 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:03.886046 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:03.902456 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc44ce8-8931-42ea-a32e-5342b225b37b None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:03.965467 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb834d3-a1f5-405b-96dd-f5bc304a8bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:03.966382 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb834d3-a1f5-405b-96dd-f5bc304a8bdd None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:03.966965 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb834d3-a1f5-405b-96dd-f5bc304a8bdd None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:03.968438 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb834d3-a1f5-405b-96dd-f5bc304a8bdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:03.988666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b8268e-4c6a-420b-bdc3-46282e84258c None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:04.047541 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b1d1a7-f5d9-4477-8b4a-752af12a0c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:04.049052 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1d1a7-f5d9-4477-8b4a-752af12a0c7a None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:04.049482 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1d1a7-f5d9-4477-8b4a-752af12a0c7a None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:04.050000 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1d1a7-f5d9-4477-8b4a-752af12a0c7a None admin] PATH_INFO: `/v3/users/8b6466e9d2284acfb0a0d945a895c9bd` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:04.055820 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b1d1a7-f5d9-4477-8b4a-752af12a0c7a None admin] RBAC: Authorizing `identity:get_user(user_id=8b6466e9d2284acfb0a0d945a895c9bd)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:04.057682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b1d1a7-f5d9-4477-8b4a-752af12a0c7a None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:04.059710 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 131/263] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:34:03 2022] GET /identity/v3/users/8b6466e9d2284acfb0a0d945a895c9bd => generated 283 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:04.065677 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b1d1a7-f5d9-4477-8b4a-752af12a0c7a None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:04.124198 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb834d3-a1f5-405b-96dd-f5bc304a8bdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:04.125701 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb834d3-a1f5-405b-96dd-f5bc304a8bdd None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:04.273787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b1d1a7-f5d9-4477-8b4a-752af12a0c7a None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:04.280271 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:04.377286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca6cb1-c9a7-4bbe-9cb9-8c4de633c33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:04.377975 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca6cb1-c9a7-4bbe-9cb9-8c4de633c33a None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:04.378256 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca6cb1-c9a7-4bbe-9cb9-8c4de633c33a None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:04.378538 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca6cb1-c9a7-4bbe-9cb9-8c4de633c33a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:04.384460 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:04.406354 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 133/264] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:34:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 714 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 14:34:04.423673 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb834d3-a1f5-405b-96dd-f5bc304a8bdd None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:04.517315 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca6cb1-c9a7-4bbe-9cb9-8c4de633c33a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:04.519085 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca6cb1-c9a7-4bbe-9cb9-8c4de633c33a None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:04.663761 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb834d3-a1f5-405b-96dd-f5bc304a8bdd None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:04.720912 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8cb3f-50b4-4c05-9a13-b66e90274dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:04.721833 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8cb3f-50b4-4c05-9a13-b66e90274dbc None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:04.722119 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8cb3f-50b4-4c05-9a13-b66e90274dbc None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:04.722396 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8cb3f-50b4-4c05-9a13-b66e90274dbc None admin] PATH_INFO: `/v3/projects/888e6ef050b643db90e7280d61f3e3e8` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:04.726875 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a8cb3f-50b4-4c05-9a13-b66e90274dbc None admin] RBAC: Authorizing `identity:get_project(project_id=888e6ef050b643db90e7280d61f3e3e8)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:04.728446 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a8cb3f-50b4-4c05-9a13-b66e90274dbc None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:04.730722 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 134/265] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:34:04 2022] GET /identity/v3/projects/888e6ef050b643db90e7280d61f3e3e8 => generated 302 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:34:04.740678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8cb3f-50b4-4c05-9a13-b66e90274dbc None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:04.748799 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 132/266] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:34:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 683 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:34:04.754490 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca6cb1-c9a7-4bbe-9cb9-8c4de633c33a None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:04.759607 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:04.835445 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:04.911388 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8cb3f-50b4-4c05-9a13-b66e90274dbc None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:04.979817 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca6cb1-c9a7-4bbe-9cb9-8c4de633c33a None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:04.990204 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b323727-296f-4536-89dd-454e0da7470d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:04.990894 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b323727-296f-4536-89dd-454e0da7470d None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:04.991206 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b323727-296f-4536-89dd-454e0da7470d None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:04.991495 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b323727-296f-4536-89dd-454e0da7470d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:04.992389 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b323727-296f-4536-89dd-454e0da7470d None admin] RBAC: Adding query filter params (role.id=49bd5abf255048a0abaa9538e309f516, scope.project.id=888e6ef050b643db90e7280d61f3e3e8, user.id=8b6466e9d2284acfb0a0d945a895c9bd) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:04.992701 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b323727-296f-4536-89dd-454e0da7470d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:04.994798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b323727-296f-4536-89dd-454e0da7470d None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:05.008916 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 135/267] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:34:04 2022] GET /identity/v3/role_assignments?user.id=8b6466e9d2284acfb0a0d945a895c9bd&scope.project.id=888e6ef050b643db90e7280d61f3e3e8&role.id=49bd5abf255048a0abaa9538e309f516 => generated 608 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:34:05.056670 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4c8e3-34be-4dcd-9df8-ef6975690cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:05.057604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4c8e3-34be-4dcd-9df8-ef6975690cdb None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:05.057889 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4c8e3-34be-4dcd-9df8-ef6975690cdb None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:05.058165 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4c8e3-34be-4dcd-9df8-ef6975690cdb None admin] PATH_INFO: `/v3/projects/888e6ef050b643db90e7280d61f3e3e8` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:05.059560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf4c8e3-34be-4dcd-9df8-ef6975690cdb None admin] RBAC: Authorizing `identity:get_project(project_id=888e6ef050b643db90e7280d61f3e3e8)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:05.061024 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf4c8e3-34be-4dcd-9df8-ef6975690cdb None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:05.062796 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 133/268] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:34:04 2022] GET /identity/v3/projects/888e6ef050b643db90e7280d61f3e3e8 => generated 302 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:34:05.069239 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b323727-296f-4536-89dd-454e0da7470d None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:05.220800 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b323727-296f-4536-89dd-454e0da7470d None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:05.258387 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:05.267408 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad2b4e-8976-4628-8800-4f4f1966617c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:05.271992 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad2b4e-8976-4628-8800-4f4f1966617c None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:05.272292 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad2b4e-8976-4628-8800-4f4f1966617c None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:05.272556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad2b4e-8976-4628-8800-4f4f1966617c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:05.273496 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ad2b4e-8976-4628-8800-4f4f1966617c None admin] RBAC: Adding query filter params (role.id=b74189174ce24c1382712858dc35f3ce, scope.project.id=888e6ef050b643db90e7280d61f3e3e8, user.id=8b6466e9d2284acfb0a0d945a895c9bd) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:05.273814 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ad2b4e-8976-4628-8800-4f4f1966617c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:05.275346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ad2b4e-8976-4628-8800-4f4f1966617c None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:05.282035 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 136/269] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:34:05 2022] GET /identity/v3/role_assignments?user.id=8b6466e9d2284acfb0a0d945a895c9bd&scope.project.id=888e6ef050b643db90e7280d61f3e3e8&role.id=b74189174ce24c1382712858dc35f3ce => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:05.306726 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9cf2c-c360-4ace-b4d8-de3f4a12e187 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:05.307208 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9cf2c-c360-4ace-b4d8-de3f4a12e187 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:05.307633 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9cf2c-c360-4ace-b4d8-de3f4a12e187 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:05.310095 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 134/270] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:05.316410 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bbee4-d314-48b0-a9a5-6e62ffee5a9d None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:05.317004 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bbee4-d314-48b0-a9a5-6e62ffee5a9d None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:05.317460 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bbee4-d314-48b0-a9a5-6e62ffee5a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:05.347013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-3e2bbee4-d314-48b0-a9a5-6e62ffee5a9d None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:05.348658 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-3e2bbee4-d314-48b0-a9a5-6e62ffee5a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:05.424055 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea4a23-a6c4-4054-884f-c6137ff8631e None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:05.424355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea4a23-a6c4-4054-884f-c6137ff8631e None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:05.424635 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea4a23-a6c4-4054-884f-c6137ff8631e None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:05.430532 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 135/271] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:05.432147 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825bd68-4535-476e-b589-67e41bfd32ba None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:05.432440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825bd68-4535-476e-b589-67e41bfd32ba None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:05.432727 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825bd68-4535-476e-b589-67e41bfd32ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:05.443137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 137/272] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:05.480389 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-d825bd68-4535-476e-b589-67e41bfd32ba None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:05.481915 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-d825bd68-4535-476e-b589-67e41bfd32ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:05.528153 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2bbee4-d314-48b0-a9a5-6e62ffee5a9d None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:05.587721 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 136/273] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:05.613724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d825bd68-4535-476e-b589-67e41bfd32ba None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:05.617117 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:05.727189 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2bbee4-d314-48b0-a9a5-6e62ffee5a9d None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:05.737305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d825bd68-4535-476e-b589-67e41bfd32ba None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:05.759499 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:05.769194 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ae1f9-5670-4199-8b7f-d7b52f2cc16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:05.770254 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ae1f9-5670-4199-8b7f-d7b52f2cc16c None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:05.770721 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ae1f9-5670-4199-8b7f-d7b52f2cc16c None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:05.771068 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ae1f9-5670-4199-8b7f-d7b52f2cc16c None admin] PATH_INFO: `/v3/roles/member` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:05.776024 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded0e1b-b005-46ae-81d3-7728a884ded3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:05.776863 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded0e1b-b005-46ae-81d3-7728a884ded3 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:05.777166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded0e1b-b005-46ae-81d3-7728a884ded3 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:05.777460 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded0e1b-b005-46ae-81d3-7728a884ded3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:05.778672 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ded0e1b-b005-46ae-81d3-7728a884ded3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:05.780433 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ded0e1b-b005-46ae-81d3-7728a884ded3 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:05.782272 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 137/274] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:05.783566 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58f7a8-1fcc-407e-a7c0-6fafd798509f None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:05.783846 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58f7a8-1fcc-407e-a7c0-6fafd798509f None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:05.784317 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58f7a8-1fcc-407e-a7c0-6fafd798509f None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:05.785644 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 138/275] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:05.788022 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58f7a8-1fcc-407e-a7c0-6fafd798509f None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:05.790451 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ae1f9-5670-4199-8b7f-d7b52f2cc16c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:05.798506 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-804ae1f9-5670-4199-8b7f-d7b52f2cc16c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:34:05.798506 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:05.798506 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:05.798506 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:05.798506 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:05.798506 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:05.798506 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:05.798506 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:05.798506 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:05.798506 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:05.798506 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:05.798506 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:05.798506 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:05.798506 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:05.798506 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:05.798506 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:05.799873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:05.799873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:05.799873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:05.799873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:05.799873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:05.799873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:05.799873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:05.799873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:05.799873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:05.799873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:05.799873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:05.799873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:05.799873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:05.799873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:05.799873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:05.799873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:05.800667 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:05.800667 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:05.800667 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:05.800667 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:05.800667 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:05.800667 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:05.800667 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:05.800667 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:05.800667 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:05.800667 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:05.800667 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:34:05.800667 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:05.800667 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 138/276] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:34:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:34:05.802230 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0834f4-2731-4d04-8eb4-a9a44d631477 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:05.802230 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0834f4-2731-4d04-8eb4-a9a44d631477 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:05.803000 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0834f4-2731-4d04-8eb4-a9a44d631477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:05.820284 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-8d0834f4-2731-4d04-8eb4-a9a44d631477 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:05.821597 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-8d0834f4-2731-4d04-8eb4-a9a44d631477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:05.882211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:05.895269 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 139/277] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:05.908072 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0834f4-2731-4d04-8eb4-a9a44d631477 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:05.933850 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58f7a8-1fcc-407e-a7c0-6fafd798509f None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:05.970796 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a5f272-764a-4f79-b07c-77b42dd8e0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:05.971773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a5f272-764a-4f79-b07c-77b42dd8e0f3 None admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:05.971885 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a5f272-764a-4f79-b07c-77b42dd8e0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:05.972161 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a5f272-764a-4f79-b07c-77b42dd8e0f3 None admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:05.972898 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a5f272-764a-4f79-b07c-77b42dd8e0f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:05.974363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a5f272-764a-4f79-b07c-77b42dd8e0f3 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:05.997777 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 139/278] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jul 20 14:34:05 2022] POST /identity/v3/users => generated 293 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:34:05.999269 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a5f272-764a-4f79-b07c-77b42dd8e0f3 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:06.074006 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0834f4-2731-4d04-8eb4-a9a44d631477 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:06.095373 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:06.121008 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de18a1-17d7-45c8-b5e3-0c568de7f236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:06.121710 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de18a1-17d7-45c8-b5e3-0c568de7f236 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:06.121996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de18a1-17d7-45c8-b5e3-0c568de7f236 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:06.122274 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de18a1-17d7-45c8-b5e3-0c568de7f236 None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:06.123089 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55de18a1-17d7-45c8-b5e3-0c568de7f236 None admin] RBAC: Adding query filter params (name=member) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:06.123403 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55de18a1-17d7-45c8-b5e3-0c568de7f236 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:06.124878 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55de18a1-17d7-45c8-b5e3-0c568de7f236 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:06.126486 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a5f272-764a-4f79-b07c-77b42dd8e0f3 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:06.134513 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 140/279] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jul 20 14:34:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:06.136283 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a552fc2-cd58-4f40-be8a-97571bdb73db None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:06.136598 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a552fc2-cd58-4f40-be8a-97571bdb73db None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:06.136879 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a552fc2-cd58-4f40-be8a-97571bdb73db None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:06.138478 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 141/280] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:06.143411 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a552fc2-cd58-4f40-be8a-97571bdb73db None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:06.163887 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:06.179501 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768bdfff-3114-4ced-9523-58acf3bd1b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:06.180305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bdfff-3114-4ced-9523-58acf3bd1b68 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:06.180591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bdfff-3114-4ced-9523-58acf3bd1b68 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:06.180857 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bdfff-3114-4ced-9523-58acf3bd1b68 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:06.193555 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768bdfff-3114-4ced-9523-58acf3bd1b68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:06.202754 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-768bdfff-3114-4ced-9523-58acf3bd1b68 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 14:34:06.202754 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:06.202754 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:06.202754 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:06.202754 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:06.202754 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:06.202754 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:06.202754 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:06.202754 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:06.202754 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:06.202754 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:06.202754 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:06.202754 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:06.202754 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:06.202754 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:06.202754 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:06.203734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:06.203734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:06.203734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:06.203734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:06.203734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:06.203734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:06.203734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:06.203734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:06.203734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:06.203734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:06.203734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:06.203734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:06.203734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:06.203734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:06.203734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:06.203734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:06.204553 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:06.204553 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:06.204553 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:06.204553 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:06.204553 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:06.204553 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:06.204553 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:06.204553 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:06.204553 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:06.204553 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:06.204553 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 14:34:06.204553 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:06.204553 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 140/281] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:34:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:06.205668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6289b-a0af-4577-9a3f-9b2a8cece349 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:06.205668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6289b-a0af-4577-9a3f-9b2a8cece349 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:06.205838 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6289b-a0af-4577-9a3f-9b2a8cece349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:06.224150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-20c6289b-a0af-4577-9a3f-9b2a8cece349 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:06.225469 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-20c6289b-a0af-4577-9a3f-9b2a8cece349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:06.289928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a552fc2-cd58-4f40-be8a-97571bdb73db None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:06.316358 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 141/282] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:06.317297 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6289b-a0af-4577-9a3f-9b2a8cece349 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:06.334976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59affde-1eae-45af-9601-c8e62ef447a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:06.337948 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59affde-1eae-45af-9601-c8e62ef447a9 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:06.338950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59affde-1eae-45af-9601-c8e62ef447a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:06.340144 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59affde-1eae-45af-9601-c8e62ef447a9 None admin] PATH_INFO: `/v3/users/d533bae8078a47a2b75ceeed7f852760` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:06.365522 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59affde-1eae-45af-9601-c8e62ef447a9 None admin] RBAC: Authorizing `identity:get_user(user_id=d533bae8078a47a2b75ceeed7f852760)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:06.367158 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59affde-1eae-45af-9601-c8e62ef447a9 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:06.368950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 142/283] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:34:06 2022] GET /identity/v3/users/d533bae8078a47a2b75ceeed7f852760 => generated 293 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:06.369755 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59affde-1eae-45af-9601-c8e62ef447a9 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:06.411327 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:06.443529 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6289b-a0af-4577-9a3f-9b2a8cece349 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:06.463484 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:06.492509 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59affde-1eae-45af-9601-c8e62ef447a9 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:06.497353 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d763e20-4d55-4b6e-85d3-13eea558498b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:06.498138 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d763e20-4d55-4b6e-85d3-13eea558498b None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:06.498487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d763e20-4d55-4b6e-85d3-13eea558498b None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:06.498825 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d763e20-4d55-4b6e-85d3-13eea558498b None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:06.499717 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d763e20-4d55-4b6e-85d3-13eea558498b None admin] RBAC: Adding query filter params (name=reader) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:06.500128 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d763e20-4d55-4b6e-85d3-13eea558498b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:06.501635 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d763e20-4d55-4b6e-85d3-13eea558498b None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:06.516785 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 142/284] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jul 20 14:34:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:06.516785 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d763e20-4d55-4b6e-85d3-13eea558498b None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:06.547263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58feb26b-4f56-44ef-9285-2bfcbc185c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:06.548600 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58feb26b-4f56-44ef-9285-2bfcbc185c90 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:06.549482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58feb26b-4f56-44ef-9285-2bfcbc185c90 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:06.549965 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58feb26b-4f56-44ef-9285-2bfcbc185c90 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:06.562927 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58feb26b-4f56-44ef-9285-2bfcbc185c90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:06.571481 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-58feb26b-4f56-44ef-9285-2bfcbc185c90 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:34:06.571481 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:06.571481 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:06.571481 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:06.571481 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:06.571481 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:06.571481 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:06.571481 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:06.571481 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:06.571481 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:06.571481 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:06.571481 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:06.571481 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:06.571481 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:06.571481 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:06.571481 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:06.572420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:06.572420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:06.572420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:06.572420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:06.572420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:06.572420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:06.572420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:06.572420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:06.572420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:06.572420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:06.572420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:06.572420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:06.572420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:06.572420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:06.572420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:06.572420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:06.573455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:06.573455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:06.573455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:06.573455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:06.573455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:06.573455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:06.573455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:06.573455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:06.573455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:06.573455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:06.573455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:34:06.573455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:06.576157 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 143/285] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:34:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:06.577269 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58feb26b-4f56-44ef-9285-2bfcbc185c90 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:06.646318 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d763e20-4d55-4b6e-85d3-13eea558498b None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:06.685070 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76796478-240f-4db8-8c34-7c839b90a2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:06.685757 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76796478-240f-4db8-8c34-7c839b90a2da None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:06.686037 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76796478-240f-4db8-8c34-7c839b90a2da None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:06.686311 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76796478-240f-4db8-8c34-7c839b90a2da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:06.687233 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76796478-240f-4db8-8c34-7c839b90a2da None admin] RBAC: Adding query filter params (role.id=7e394dd183b147ffa15b617f076bbb76, user.id=d533bae8078a47a2b75ceeed7f852760, scope.system=all) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:06.687518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76796478-240f-4db8-8c34-7c839b90a2da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:06.689031 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76796478-240f-4db8-8c34-7c839b90a2da None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:06.694706 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 143/286] 10.222.0.15 () {56 vars in 1308 bytes} [Wed Jul 20 14:34:06 2022] GET /identity/v3/role_assignments?user.id=d533bae8078a47a2b75ceeed7f852760&scope.system=all&role.id=7e394dd183b147ffa15b617f076bbb76 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:06.699544 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76796478-240f-4db8-8c34-7c839b90a2da None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:06.714784 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:06.729570 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58feb26b-4f56-44ef-9285-2bfcbc185c90 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:06.784949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16035d00-3ec0-4dc4-acf0-fb9a2e10f4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:06.786252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16035d00-3ec0-4dc4-acf0-fb9a2e10f4cd None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:06.789030 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16035d00-3ec0-4dc4-acf0-fb9a2e10f4cd None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:06.789636 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16035d00-3ec0-4dc4-acf0-fb9a2e10f4cd None admin] PATH_INFO: `/v3/users/717529633938498bb7dd18cfeff4989f` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:06.791403 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16035d00-3ec0-4dc4-acf0-fb9a2e10f4cd None admin] RBAC: Authorizing `identity:get_user(user_id=717529633938498bb7dd18cfeff4989f)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:06.793808 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16035d00-3ec0-4dc4-acf0-fb9a2e10f4cd None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:06.797800 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:06.802747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 144/287] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:34:06 2022] GET /identity/v3/users/717529633938498bb7dd18cfeff4989f => generated 297 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:06.804834 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16035d00-3ec0-4dc4-acf0-fb9a2e10f4cd None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:06.917070 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76796478-240f-4db8-8c34-7c839b90a2da None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:06.959700 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ad8a1-aefe-4ae2-8bd8-bd36f93f956e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:06.960385 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ad8a1-aefe-4ae2-8bd8-bd36f93f956e None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:06.960725 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ad8a1-aefe-4ae2-8bd8-bd36f93f956e None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:06.961003 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ad8a1-aefe-4ae2-8bd8-bd36f93f956e None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:06.961756 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ad8a1-aefe-4ae2-8bd8-bd36f93f956e None admin] RBAC: Adding query filter params (name=member) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:06.962061 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ad8a1-aefe-4ae2-8bd8-bd36f93f956e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:06.971716 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ad8a1-aefe-4ae2-8bd8-bd36f93f956e None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:06.973619 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16035d00-3ec0-4dc4-acf0-fb9a2e10f4cd None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:06.990718 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 144/288] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jul 20 14:34:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:06.996110 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ad8a1-aefe-4ae2-8bd8-bd36f93f956e None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:07.021601 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd9bdc-fa48-47f1-bfe0-ca12c808b08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:07.022483 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd9bdc-fa48-47f1-bfe0-ca12c808b08f None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:07.023939 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd9bdc-fa48-47f1-bfe0-ca12c808b08f None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:07.024146 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd9bdc-fa48-47f1-bfe0-ca12c808b08f None admin] PATH_INFO: `/v3/projects/888e6ef050b643db90e7280d61f3e3e8` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:07.027292 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd9bdc-fa48-47f1-bfe0-ca12c808b08f None admin] RBAC: Authorizing `identity:get_project(project_id=888e6ef050b643db90e7280d61f3e3e8)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:07.029518 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd9bdc-fa48-47f1-bfe0-ca12c808b08f None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:07.035327 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 145/289] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:34:06 2022] GET /identity/v3/projects/888e6ef050b643db90e7280d61f3e3e8 => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:07.037835 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd9bdc-fa48-47f1-bfe0-ca12c808b08f None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:07.071709 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:07.148790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd9bdc-fa48-47f1-bfe0-ca12c808b08f None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:07.194221 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ec8aa-d5cf-4fa5-a593-82781540f107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:07.195200 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ec8aa-d5cf-4fa5-a593-82781540f107 None admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:07.195660 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ec8aa-d5cf-4fa5-a593-82781540f107 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:07.195979 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ec8aa-d5cf-4fa5-a593-82781540f107 None admin] PATH_INFO: `/v3/projects/888e6ef050b643db90e7280d61f3e3e8/users/717529633938498bb7dd18cfeff4989f/roles/483f28cfc7444513a119d5d363f64518` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:07.198237 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ec8aa-d5cf-4fa5-a593-82781540f107 None admin] RBAC: Authorizing `identity:create_grant(project_id=888e6ef050b643db90e7280d61f3e3e8, user_id=717529633938498bb7dd18cfeff4989f, role_id=483f28cfc7444513a119d5d363f64518)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:07.200174 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ec8aa-d5cf-4fa5-a593-82781540f107 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:07.224770 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 146/290] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jul 20 14:34:07 2022] PUT /identity/v3/projects/888e6ef050b643db90e7280d61f3e3e8/users/717529633938498bb7dd18cfeff4989f/roles/483f28cfc7444513a119d5d363f64518 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:34:07.255754 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ad8a1-aefe-4ae2-8bd8-bd36f93f956e None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:07.265707 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:07.318434 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041893a6-9064-4d97-afc9-3a2a909f1ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:07.319390 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041893a6-9064-4d97-afc9-3a2a909f1ece None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:07.319680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041893a6-9064-4d97-afc9-3a2a909f1ece None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:07.320015 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041893a6-9064-4d97-afc9-3a2a909f1ece None admin] PATH_INFO: `/v3/users/68c8c7a1c47147c6b4956416370e3e2a` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:07.324028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041893a6-9064-4d97-afc9-3a2a909f1ece None admin] RBAC: Authorizing `identity:get_user(user_id=68c8c7a1c47147c6b4956416370e3e2a)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:07.325566 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041893a6-9064-4d97-afc9-3a2a909f1ece None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:07.327803 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 145/291] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:34:06 2022] GET /identity/v3/users/68c8c7a1c47147c6b4956416370e3e2a => generated 297 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:34:07.333485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ec8aa-d5cf-4fa5-a593-82781540f107 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:07.441822 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:07.455669 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ec8aa-d5cf-4fa5-a593-82781540f107 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:07.493821 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74f2a4-0e02-45a7-bcec-e2005ee31a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:07.494650 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74f2a4-0e02-45a7-bcec-e2005ee31a35 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:07.494943 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74f2a4-0e02-45a7-bcec-e2005ee31a35 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:07.495259 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74f2a4-0e02-45a7-bcec-e2005ee31a35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:07.571710 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74f2a4-0e02-45a7-bcec-e2005ee31a35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:07.573320 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74f2a4-0e02-45a7-bcec-e2005ee31a35 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:07.690229 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:07.704751 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 147/292] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:34:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:34:07.710597 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041893a6-9064-4d97-afc9-3a2a909f1ece None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:07.859904 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041893a6-9064-4d97-afc9-3a2a909f1ece None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:07.905231 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69989ac5-4c0a-4db5-b5ed-a87cb8307735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:07.906371 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69989ac5-4c0a-4db5-b5ed-a87cb8307735 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:07.906796 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69989ac5-4c0a-4db5-b5ed-a87cb8307735 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:07.907241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69989ac5-4c0a-4db5-b5ed-a87cb8307735 None admin] PATH_INFO: `/v3/projects/888e6ef050b643db90e7280d61f3e3e8` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:07.908640 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69989ac5-4c0a-4db5-b5ed-a87cb8307735 None admin] RBAC: Authorizing `identity:get_project(project_id=888e6ef050b643db90e7280d61f3e3e8)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:07.910246 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69989ac5-4c0a-4db5-b5ed-a87cb8307735 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:07.912172 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 146/293] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:34:07 2022] GET /identity/v3/projects/888e6ef050b643db90e7280d61f3e3e8 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:07.918266 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74f2a4-0e02-45a7-bcec-e2005ee31a35 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:08.074019 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74f2a4-0e02-45a7-bcec-e2005ee31a35 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:08.117788 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16266e3-f0ae-448c-bc6a-b5be7fddc743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:08.118474 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16266e3-f0ae-448c-bc6a-b5be7fddc743 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:08.118764 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16266e3-f0ae-448c-bc6a-b5be7fddc743 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:08.119063 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16266e3-f0ae-448c-bc6a-b5be7fddc743 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:08.120180 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16266e3-f0ae-448c-bc6a-b5be7fddc743 None admin] RBAC: Adding query filter params (role.id=7e394dd183b147ffa15b617f076bbb76, scope.project.id=888e6ef050b643db90e7280d61f3e3e8, user.id=68c8c7a1c47147c6b4956416370e3e2a) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:08.120500 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16266e3-f0ae-448c-bc6a-b5be7fddc743 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:08.121987 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16266e3-f0ae-448c-bc6a-b5be7fddc743 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:08.134802 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 148/294] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:34:07 2022] GET /identity/v3/role_assignments?user.id=68c8c7a1c47147c6b4956416370e3e2a&scope.project.id=888e6ef050b643db90e7280d61f3e3e8&role.id=7e394dd183b147ffa15b617f076bbb76 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:08.532751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c57b80-348b-4903-935f-6982f19df939 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:08.533451 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c57b80-348b-4903-935f-6982f19df939 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:08.533930 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c57b80-348b-4903-935f-6982f19df939 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:08.535563 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 147/295] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:08.541207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd117cb7-50d9-497c-af15-a4a8f3a406d9 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:08.542096 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd117cb7-50d9-497c-af15-a4a8f3a406d9 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:08.542604 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd117cb7-50d9-497c-af15-a4a8f3a406d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:08.562547 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-fd117cb7-50d9-497c-af15-a4a8f3a406d9 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:08.564185 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-fd117cb7-50d9-497c-af15-a4a8f3a406d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:08.624429 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:08.647666 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 149/296] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:08.655973 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ebfdc-4ff6-457f-b02a-a37cb4187a95 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:08.656269 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ebfdc-4ff6-457f-b02a-a37cb4187a95 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:08.656557 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ebfdc-4ff6-457f-b02a-a37cb4187a95 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:08.658294 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 148/297] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:08.663999 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e75b9-e98c-42ca-991d-be5f67e3bca5 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:08.664643 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e75b9-e98c-42ca-991d-be5f67e3bca5 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:08.665156 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e75b9-e98c-42ca-991d-be5f67e3bca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:08.670061 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557ebfdc-4ff6-457f-b02a-a37cb4187a95 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:08.683621 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-e59e75b9-e98c-42ca-991d-be5f67e3bca5 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:08.684849 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-e59e75b9-e98c-42ca-991d-be5f67e3bca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:08.725095 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:08.772090 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 150/298] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:08.811326 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e75b9-e98c-42ca-991d-be5f67e3bca5 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:08.818247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557ebfdc-4ff6-457f-b02a-a37cb4187a95 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:08.876174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ff8b9-646b-4410-bc60-85a1a7c18a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:08.877009 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ff8b9-646b-4410-bc60-85a1a7c18a97 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:08.877295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ff8b9-646b-4410-bc60-85a1a7c18a97 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:08.877571 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ff8b9-646b-4410-bc60-85a1a7c18a97 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:08.878886 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ff8b9-646b-4410-bc60-85a1a7c18a97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:08.880927 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ff8b9-646b-4410-bc60-85a1a7c18a97 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:08.887098 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 149/299] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:08.894460 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ff8b9-646b-4410-bc60-85a1a7c18a97 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:08.961339 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e75b9-e98c-42ca-991d-be5f67e3bca5 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:08.983001 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:09.015138 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28662cfc-140f-4df4-89a2-b1fb3cd531df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:09.016465 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28662cfc-140f-4df4-89a2-b1fb3cd531df None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:09.016894 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28662cfc-140f-4df4-89a2-b1fb3cd531df None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:09.017701 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28662cfc-140f-4df4-89a2-b1fb3cd531df None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:09.026967 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ff8b9-646b-4410-bc60-85a1a7c18a97 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:09.033934 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28662cfc-140f-4df4-89a2-b1fb3cd531df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:09.038235 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:09.047733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-28662cfc-140f-4df4-89a2-b1fb3cd531df None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 14:34:09.047733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:09.047733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:09.047733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:09.047733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:09.047733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:09.047733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:09.047733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:09.047733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:09.047733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:09.047733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:09.047733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:09.047733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:09.047733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:09.047733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:09.047733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:09.049270 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:09.049270 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:09.049270 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:09.049270 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:09.049270 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:09.049270 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:09.049270 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:09.049270 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:09.049270 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:09.049270 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:09.049270 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:09.049270 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:09.049270 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:09.049270 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:09.049270 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:09.049270 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:09.050253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:09.050253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:09.050253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:09.050253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:09.050253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:09.050253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:09.050253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:09.050253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:09.050253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:09.050253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:09.050253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 14:34:09.050253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:09.059199 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 151/300] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:34:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:09.059804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28662cfc-140f-4df4-89a2-b1fb3cd531df None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:09.075607 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b41df50-a12f-4080-b0c1-ebf3892ff1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:09.076299 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41df50-a12f-4080-b0c1-ebf3892ff1a4 None admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:09.076575 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41df50-a12f-4080-b0c1-ebf3892ff1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:09.077214 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41df50-a12f-4080-b0c1-ebf3892ff1a4 None admin] PATH_INFO: `/v3/groups` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:09.077644 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b41df50-a12f-4080-b0c1-ebf3892ff1a4 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:09.079226 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b41df50-a12f-4080-b0c1-ebf3892ff1a4 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:09.097494 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 150/301] 10.222.0.15 () {60 vars in 1134 bytes} [Wed Jul 20 14:34:08 2022] POST /identity/v3/groups => generated 229 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 14:34:09.190120 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28662cfc-140f-4df4-89a2-b1fb3cd531df None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:09.231528 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a85401-6271-40e1-bb44-6eba86a6de75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:09.232483 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a85401-6271-40e1-bb44-6eba86a6de75 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:09.232913 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a85401-6271-40e1-bb44-6eba86a6de75 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:09.233339 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a85401-6271-40e1-bb44-6eba86a6de75 None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:09.234291 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a85401-6271-40e1-bb44-6eba86a6de75 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:09.234743 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a85401-6271-40e1-bb44-6eba86a6de75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:09.237847 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a85401-6271-40e1-bb44-6eba86a6de75 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:09.248912 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 152/302] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jul 20 14:34:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:09.254966 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b41df50-a12f-4080-b0c1-ebf3892ff1a4 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:09.261254 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbaff0-dd58-4a46-bbd7-a6d93bcd047f None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:09.261542 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbaff0-dd58-4a46-bbd7-a6d93bcd047f None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:09.261823 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbaff0-dd58-4a46-bbd7-a6d93bcd047f None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:09.263614 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 153/303] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 14:34:09.268944 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aecd14-14d3-4de1-be23-a3d3d08ac7f4 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:09.269234 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aecd14-14d3-4de1-be23-a3d3d08ac7f4 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:09.269515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aecd14-14d3-4de1-be23-a3d3d08ac7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:09.288479 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-a0aecd14-14d3-4de1-be23-a3d3d08ac7f4 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:09.290196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-a0aecd14-14d3-4de1-be23-a3d3d08ac7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:09.321977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:09.373296 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 154/304] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:09.399749 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aecd14-14d3-4de1-be23-a3d3d08ac7f4 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:09.400434 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b41df50-a12f-4080-b0c1-ebf3892ff1a4 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:09.444950 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ca053-a83f-4d0a-a709-525994bcfee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:09.446744 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ca053-a83f-4d0a-a709-525994bcfee9 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:09.447626 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ca053-a83f-4d0a-a709-525994bcfee9 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:09.448509 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ca053-a83f-4d0a-a709-525994bcfee9 None admin] PATH_INFO: `/v3/users/e356ef9e492c4ec981a6bbb5f5fbe188` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:09.468361 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91ca053-a83f-4d0a-a709-525994bcfee9 None admin] RBAC: Authorizing `identity:get_user(user_id=e356ef9e492c4ec981a6bbb5f5fbe188)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:09.470499 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91ca053-a83f-4d0a-a709-525994bcfee9 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:09.473442 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 151/305] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:34:09 2022] GET /identity/v3/users/e356ef9e492c4ec981a6bbb5f5fbe188 => generated 293 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:09.481944 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ca053-a83f-4d0a-a709-525994bcfee9 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:09.520865 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:09.524852 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aecd14-14d3-4de1-be23-a3d3d08ac7f4 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:09.568181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6750e80-01b6-453f-9708-3d5119f92b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:09.569698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6750e80-01b6-453f-9708-3d5119f92b53 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:09.570190 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6750e80-01b6-453f-9708-3d5119f92b53 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:09.570635 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6750e80-01b6-453f-9708-3d5119f92b53 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:09.582440 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6750e80-01b6-453f-9708-3d5119f92b53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:09.591352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-f6750e80-01b6-453f-9708-3d5119f92b53 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:34:09.591352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:09.591352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:09.591352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:09.591352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:09.591352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:09.591352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:09.591352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:09.591352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:09.591352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:09.591352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:09.591352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:09.591352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:09.591352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:09.591352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:09.591352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:09.592381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:09.592381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:09.592381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:09.592381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:09.592381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:09.592381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:09.592381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:09.592381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:09.592381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:09.592381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:09.592381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:09.592381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:09.592381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:09.592381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:09.592381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:09.592381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:09.593161 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:09.593161 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:09.593161 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:09.593161 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:09.593161 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:09.593161 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:09.593161 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:09.593161 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:09.593161 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:09.593161 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:09.593161 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:34:09.593161 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:09.599316 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 155/306] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:34:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:34:09.602200 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6750e80-01b6-453f-9708-3d5119f92b53 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:09.620572 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ca053-a83f-4d0a-a709-525994bcfee9 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:09.635460 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:09.662816 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf114be-1198-421b-b55a-ad134ec28567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:09.663546 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf114be-1198-421b-b55a-ad134ec28567 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:09.663901 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf114be-1198-421b-b55a-ad134ec28567 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:09.664183 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf114be-1198-421b-b55a-ad134ec28567 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:09.665053 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf114be-1198-421b-b55a-ad134ec28567 None admin] RBAC: Adding query filter params (role.id=483f28cfc7444513a119d5d363f64518, user.id=e356ef9e492c4ec981a6bbb5f5fbe188, scope.system=all) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:09.665361 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf114be-1198-421b-b55a-ad134ec28567 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:09.666957 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf114be-1198-421b-b55a-ad134ec28567 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:09.673362 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 152/307] 10.222.0.15 () {56 vars in 1308 bytes} [Wed Jul 20 14:34:09 2022] GET /identity/v3/role_assignments?user.id=e356ef9e492c4ec981a6bbb5f5fbe188&scope.system=all&role.id=483f28cfc7444513a119d5d363f64518 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:09.724205 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6750e80-01b6-453f-9708-3d5119f92b53 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:09.765115 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181c7ac-d3eb-4c35-81f9-47e9525ed8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:09.766093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181c7ac-d3eb-4c35-81f9-47e9525ed8ae None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:09.766524 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181c7ac-d3eb-4c35-81f9-47e9525ed8ae None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:09.766973 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181c7ac-d3eb-4c35-81f9-47e9525ed8ae None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:09.767929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181c7ac-d3eb-4c35-81f9-47e9525ed8ae None admin] RBAC: Adding query filter params (name=member) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:09.768390 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181c7ac-d3eb-4c35-81f9-47e9525ed8ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:09.769971 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181c7ac-d3eb-4c35-81f9-47e9525ed8ae None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:09.780623 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 156/308] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jul 20 14:34:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:09.786663 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf114be-1198-421b-b55a-ad134ec28567 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:09.900838 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:09.912278 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf114be-1198-421b-b55a-ad134ec28567 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:09.963308 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50626b10-6672-4501-adc5-d02c89e744c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:09.964227 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50626b10-6672-4501-adc5-d02c89e744c3 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:09.964511 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50626b10-6672-4501-adc5-d02c89e744c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:09.964787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50626b10-6672-4501-adc5-d02c89e744c3 None admin] PATH_INFO: `/v3/users/d533bae8078a47a2b75ceeed7f852760` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:09.966016 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50626b10-6672-4501-adc5-d02c89e744c3 None admin] RBAC: Authorizing `identity:get_user(user_id=d533bae8078a47a2b75ceeed7f852760)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:09.967535 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50626b10-6672-4501-adc5-d02c89e744c3 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:09.969554 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 153/309] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:34:09 2022] GET /identity/v3/users/d533bae8078a47a2b75ceeed7f852760 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:09.976552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181c7ac-d3eb-4c35-81f9-47e9525ed8ae None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:10.016427 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25f5f3-9878-4b18-98e8-00867940f389 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:10.017060 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25f5f3-9878-4b18-98e8-00867940f389 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:10.017537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25f5f3-9878-4b18-98e8-00867940f389 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:10.019946 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 154/310] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 14:34:10.027081 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3073880-bd62-4c91-9359-42eb2710a717 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:10.027384 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3073880-bd62-4c91-9359-42eb2710a717 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:10.027668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3073880-bd62-4c91-9359-42eb2710a717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:10.048505 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-f3073880-bd62-4c91-9359-42eb2710a717 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:10.050256 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-f3073880-bd62-4c91-9359-42eb2710a717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:10.071930 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:10.089831 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181c7ac-d3eb-4c35-81f9-47e9525ed8ae None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:10.126721 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b43376-24b1-42c2-a6ce-675e0bcff959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:10.128692 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b43376-24b1-42c2-a6ce-675e0bcff959 None admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:10.129219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b43376-24b1-42c2-a6ce-675e0bcff959 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:10.129695 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b43376-24b1-42c2-a6ce-675e0bcff959 None admin] PATH_INFO: `/v3/system/users/d533bae8078a47a2b75ceeed7f852760/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:10.131647 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b43376-24b1-42c2-a6ce-675e0bcff959 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d533bae8078a47a2b75ceeed7f852760, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:10.133376 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b43376-24b1-42c2-a6ce-675e0bcff959 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:10.139118 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 155/311] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 14:34:10.154444 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 157/312] 10.222.0.15 () {58 vars in 1334 bytes} [Wed Jul 20 14:34:09 2022] PUT /identity/v3/system/users/d533bae8078a47a2b75ceeed7f852760/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:34:10.159526 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3073880-bd62-4c91-9359-42eb2710a717 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:10.270685 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:10.290951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3073880-bd62-4c91-9359-42eb2710a717 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:10.343203 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32cca95-7fd7-4071-823b-39e4a4e405d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:10.344441 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32cca95-7fd7-4071-823b-39e4a4e405d1 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:10.344732 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32cca95-7fd7-4071-823b-39e4a4e405d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:10.345012 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32cca95-7fd7-4071-823b-39e4a4e405d1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:10.358284 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32cca95-7fd7-4071-823b-39e4a4e405d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:10.366310 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-e32cca95-7fd7-4071-823b-39e4a4e405d1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 14:34:10.366310 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:10.366310 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:10.366310 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:10.366310 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:10.366310 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:10.366310 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:10.366310 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:10.366310 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:10.366310 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:10.366310 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:10.366310 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:10.366310 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:10.366310 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:10.366310 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:10.366310 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:10.368033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:10.368033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:10.368033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:10.368033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:10.368033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:10.368033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:10.368033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:10.368033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:10.368033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:10.368033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:10.368033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:10.368033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:10.368033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:10.368033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:10.368033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:10.368033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:10.369560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:10.369560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:10.369560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:10.369560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:10.369560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:10.369560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:10.369560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:10.369560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:10.369560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:10.369560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:10.369560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 14:34:10.369560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:10.369560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 156/313] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:34:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:10.378144 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b43376-24b1-42c2-a6ce-675e0bcff959 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:10.498935 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b43376-24b1-42c2-a6ce-675e0bcff959 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:10.534772 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba9458-b220-43d5-98e7-3de7c277a0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:10.535494 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba9458-b220-43d5-98e7-3de7c277a0b1 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:10.535779 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba9458-b220-43d5-98e7-3de7c277a0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:10.536060 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba9458-b220-43d5-98e7-3de7c277a0b1 None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:10.536832 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba9458-b220-43d5-98e7-3de7c277a0b1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:10.537181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba9458-b220-43d5-98e7-3de7c277a0b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:10.538665 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba9458-b220-43d5-98e7-3de7c277a0b1 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:10.552033 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 158/314] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jul 20 14:34:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:10.557799 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32cca95-7fd7-4071-823b-39e4a4e405d1 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:10.678741 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32cca95-7fd7-4071-823b-39e4a4e405d1 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:10.717081 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c73a4-47c2-4cf9-8daa-991a69148012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:10.718353 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c73a4-47c2-4cf9-8daa-991a69148012 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:10.718876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c73a4-47c2-4cf9-8daa-991a69148012 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:10.719372 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c73a4-47c2-4cf9-8daa-991a69148012 None admin] PATH_INFO: `/v3/users/717529633938498bb7dd18cfeff4989f` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:10.720821 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c73a4-47c2-4cf9-8daa-991a69148012 None admin] RBAC: Authorizing `identity:get_user(user_id=717529633938498bb7dd18cfeff4989f)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:10.722570 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c73a4-47c2-4cf9-8daa-991a69148012 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:10.724673 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 157/315] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:34:10 2022] GET /identity/v3/users/717529633938498bb7dd18cfeff4989f => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:10.733404 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba9458-b220-43d5-98e7-3de7c277a0b1 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:10.751359 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:10.843995 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba9458-b220-43d5-98e7-3de7c277a0b1 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:10.880631 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b945b-1a14-45c3-8f70-7fbb9ecc99fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:10.881308 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b945b-1a14-45c3-8f70-7fbb9ecc99fa None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:10.881594 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b945b-1a14-45c3-8f70-7fbb9ecc99fa None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:10.881871 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b945b-1a14-45c3-8f70-7fbb9ecc99fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:10.954151 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667b945b-1a14-45c3-8f70-7fbb9ecc99fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:10.955698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667b945b-1a14-45c3-8f70-7fbb9ecc99fa None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:10.972404 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:11.063829 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 159/316] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:34:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:34:11.069372 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c73a4-47c2-4cf9-8daa-991a69148012 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:11.094705 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:11.186166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c73a4-47c2-4cf9-8daa-991a69148012 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:11.224507 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74785e-9e58-48ec-a2a6-fe06ecf5938a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:11.225785 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74785e-9e58-48ec-a2a6-fe06ecf5938a None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:11.226421 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74785e-9e58-48ec-a2a6-fe06ecf5938a None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:11.226897 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74785e-9e58-48ec-a2a6-fe06ecf5938a None admin] PATH_INFO: `/v3/projects/888e6ef050b643db90e7280d61f3e3e8` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:11.228660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae74785e-9e58-48ec-a2a6-fe06ecf5938a None admin] RBAC: Authorizing `identity:get_project(project_id=888e6ef050b643db90e7280d61f3e3e8)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:11.230344 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae74785e-9e58-48ec-a2a6-fe06ecf5938a None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:11.232663 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 158/317] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:34:11 2022] GET /identity/v3/projects/888e6ef050b643db90e7280d61f3e3e8 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:11.240468 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b945b-1a14-45c3-8f70-7fbb9ecc99fa None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:11.264198 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cee1e1-28af-46fc-93a5-a7cf87eb19b9 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:11.264434 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cee1e1-28af-46fc-93a5-a7cf87eb19b9 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:11.264719 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cee1e1-28af-46fc-93a5-a7cf87eb19b9 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:11.266375 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 159/318] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:11.271456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963cf2da-01f2-4cbf-aac0-972216a3438a None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:11.271743 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963cf2da-01f2-4cbf-aac0-972216a3438a None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:11.272019 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963cf2da-01f2-4cbf-aac0-972216a3438a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:11.290257 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-963cf2da-01f2-4cbf-aac0-972216a3438a None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:11.291870 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-963cf2da-01f2-4cbf-aac0-972216a3438a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:11.366796 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b945b-1a14-45c3-8f70-7fbb9ecc99fa None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:11.373111 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 160/319] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 14:34:11.391963 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963cf2da-01f2-4cbf-aac0-972216a3438a None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:11.395133 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:11.412725 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:11.435756 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732dc9a-c27f-4bb3-ba85-d17738cc1acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:11.436905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732dc9a-c27f-4bb3-ba85-d17738cc1acb None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:11.437423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732dc9a-c27f-4bb3-ba85-d17738cc1acb None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:11.437990 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732dc9a-c27f-4bb3-ba85-d17738cc1acb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:11.439607 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c732dc9a-c27f-4bb3-ba85-d17738cc1acb None admin] RBAC: Adding query filter params (role.id=483f28cfc7444513a119d5d363f64518, scope.project.id=888e6ef050b643db90e7280d61f3e3e8, user.id=717529633938498bb7dd18cfeff4989f) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:11.440125 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c732dc9a-c27f-4bb3-ba85-d17738cc1acb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:11.442944 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c732dc9a-c27f-4bb3-ba85-d17738cc1acb None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:11.459879 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 160/320] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:34:11 2022] GET /identity/v3/role_assignments?user.id=717529633938498bb7dd18cfeff4989f&scope.project.id=888e6ef050b643db90e7280d61f3e3e8&role.id=483f28cfc7444513a119d5d363f64518 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:11.507866 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963cf2da-01f2-4cbf-aac0-972216a3438a None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:11.545500 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbf52b-ec26-4604-92dc-7a03693a3f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:11.546672 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbf52b-ec26-4604-92dc-7a03693a3f50 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:11.547161 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbf52b-ec26-4604-92dc-7a03693a3f50 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:11.547605 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbf52b-ec26-4604-92dc-7a03693a3f50 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:11.549504 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbbf52b-ec26-4604-92dc-7a03693a3f50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:11.551227 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbbf52b-ec26-4604-92dc-7a03693a3f50 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:11.553495 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 161/321] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:34:11.559620 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732dc9a-c27f-4bb3-ba85-d17738cc1acb None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:11.677742 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732dc9a-c27f-4bb3-ba85-d17738cc1acb None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:11.715734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522aba10-b02a-44ac-b319-2fc2681e1d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:11.716729 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522aba10-b02a-44ac-b319-2fc2681e1d38 None admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:11.717205 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522aba10-b02a-44ac-b319-2fc2681e1d38 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:11.717647 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522aba10-b02a-44ac-b319-2fc2681e1d38 None admin] PATH_INFO: `/v3/groups` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:11.718517 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522aba10-b02a-44ac-b319-2fc2681e1d38 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:11.720182 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522aba10-b02a-44ac-b319-2fc2681e1d38 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:11.739828 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 161/322] 10.222.0.15 () {60 vars in 1134 bytes} [Wed Jul 20 14:34:11 2022] POST /identity/v3/groups => generated 226 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:34:11.958683 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3b6d3-f434-4913-938c-b22c47510686 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:11.959029 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3b6d3-f434-4913-938c-b22c47510686 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:11.959243 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3b6d3-f434-4913-938c-b22c47510686 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:11.960648 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 162/323] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:11.966420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf12a3a-a499-4795-b803-f198e46fd70e None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:11.966721 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf12a3a-a499-4795-b803-f198e46fd70e None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:11.966969 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf12a3a-a499-4795-b803-f198e46fd70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:11.987169 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-4cf12a3a-a499-4795-b803-f198e46fd70e None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:11.988397 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-4cf12a3a-a499-4795-b803-f198e46fd70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:12.026282 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:12.077303 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 162/324] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:12.101598 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c3b6d3-f434-4913-938c-b22c47510686 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:12.196706 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:12.223290 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c3b6d3-f434-4913-938c-b22c47510686 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:12.267899 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86c470-e946-4042-b556-eb379103a94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:12.269113 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86c470-e946-4042-b556-eb379103a94b None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:12.269532 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86c470-e946-4042-b556-eb379103a94b None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:12.269945 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86c470-e946-4042-b556-eb379103a94b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:12.288474 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c86c470-e946-4042-b556-eb379103a94b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:12.297407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-0c86c470-e946-4042-b556-eb379103a94b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 14:34:12.297407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:12.297407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:12.297407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:12.297407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:12.297407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:12.297407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:12.297407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:12.297407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:12.297407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:12.297407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:12.297407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:12.297407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:12.297407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:12.297407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:12.297407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:12.298417 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:12.298417 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:12.298417 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:12.298417 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:12.298417 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:12.298417 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:12.298417 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:12.298417 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:12.298417 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:12.298417 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:12.298417 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:12.298417 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:12.298417 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:12.298417 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:12.298417 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:12.298417 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:12.299629 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:12.299629 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:12.299629 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:12.299629 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:12.299629 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:12.299629 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:12.299629 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:12.299629 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:12.299629 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:12.299629 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:12.299629 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 14:34:12.299629 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:12.302927 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 163/325] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:34:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:12.311079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf12a3a-a499-4795-b803-f198e46fd70e None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:12.410792 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd884f-6638-40f2-9c24-9b19139709bd None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:12.411150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd884f-6638-40f2-9c24-9b19139709bd None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:12.411383 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd884f-6638-40f2-9c24-9b19139709bd None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:12.412993 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 164/326] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:12.418217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fa20a-7be1-4cf9-a054-5a472351217b None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:12.418560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fa20a-7be1-4cf9-a054-5a472351217b None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:12.418843 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fa20a-7be1-4cf9-a054-5a472351217b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:12.444318 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-cf2fa20a-7be1-4cf9-a054-5a472351217b None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:12.446105 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-cf2fa20a-7be1-4cf9-a054-5a472351217b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:12.448080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf12a3a-a499-4795-b803-f198e46fd70e None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:12.499820 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce2876-5c83-4c09-a75d-7ee7dabfd1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:12.500565 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce2876-5c83-4c09-a75d-7ee7dabfd1b4 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:12.500854 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce2876-5c83-4c09-a75d-7ee7dabfd1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:12.501133 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce2876-5c83-4c09-a75d-7ee7dabfd1b4 None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:12.501903 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce2876-5c83-4c09-a75d-7ee7dabfd1b4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:12.502214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce2876-5c83-4c09-a75d-7ee7dabfd1b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:12.503661 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce2876-5c83-4c09-a75d-7ee7dabfd1b4 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:12.519090 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 163/327] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jul 20 14:34:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:12.521629 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce2876-5c83-4c09-a75d-7ee7dabfd1b4 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:12.546523 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 165/328] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 14:34:12.568732 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:12.574834 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2fa20a-7be1-4cf9-a054-5a472351217b None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:12.655729 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce2876-5c83-4c09-a75d-7ee7dabfd1b4 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:12.675026 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:12.698488 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1625665a-b126-4c67-a966-5a8f4ebd12e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:12.699859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625665a-b126-4c67-a966-5a8f4ebd12e2 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:12.700257 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2fa20a-7be1-4cf9-a054-5a472351217b None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:12.700560 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625665a-b126-4c67-a966-5a8f4ebd12e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:12.701035 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625665a-b126-4c67-a966-5a8f4ebd12e2 None admin] PATH_INFO: `/v3/users/e356ef9e492c4ec981a6bbb5f5fbe188` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:12.702736 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1625665a-b126-4c67-a966-5a8f4ebd12e2 None admin] RBAC: Authorizing `identity:get_user(user_id=e356ef9e492c4ec981a6bbb5f5fbe188)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:12.704519 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1625665a-b126-4c67-a966-5a8f4ebd12e2 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:12.707927 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 164/329] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:34:12 2022] GET /identity/v3/users/e356ef9e492c4ec981a6bbb5f5fbe188 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:12.713977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1625665a-b126-4c67-a966-5a8f4ebd12e2 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:12.737898 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a6dee-e678-4883-a550-dd6d42fa3b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:12.738536 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a6dee-e678-4883-a550-dd6d42fa3b0f None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:12.738824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a6dee-e678-4883-a550-dd6d42fa3b0f None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:12.739680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a6dee-e678-4883-a550-dd6d42fa3b0f None admin] PATH_INFO: `/v3/roles/member` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:12.752116 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3a6dee-e678-4883-a550-dd6d42fa3b0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:12.760295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-4f3a6dee-e678-4883-a550-dd6d42fa3b0f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:34:12.760295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:12.760295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:12.760295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:12.760295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:12.760295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:12.760295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:12.760295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:12.760295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:12.760295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:12.760295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:12.760295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:12.760295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:12.760295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:12.760295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:12.760295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:12.761225 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:12.761225 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:12.761225 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:12.761225 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:12.761225 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:12.761225 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:12.761225 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:12.761225 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:12.761225 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:12.761225 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:12.761225 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:12.761225 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:12.761225 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:12.761225 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:12.761225 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:12.761225 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:12.762019 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:12.762019 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:12.762019 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:12.762019 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:12.762019 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:12.762019 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:12.762019 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:12.762019 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:12.762019 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:12.762019 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:12.762019 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:34:12.762019 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:12.767396 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 166/330] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:34:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:34:12.768347 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a6dee-e678-4883-a550-dd6d42fa3b0f None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:12.834687 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:12.846125 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1625665a-b126-4c67-a966-5a8f4ebd12e2 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:12.919846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367e61d-f0bc-44b9-aeef-b5997d17aa0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:12.924737 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367e61d-f0bc-44b9-aeef-b5997d17aa0c None admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:12.925016 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367e61d-f0bc-44b9-aeef-b5997d17aa0c None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:12.925295 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367e61d-f0bc-44b9-aeef-b5997d17aa0c None admin] PATH_INFO: `/v3/system/users/e356ef9e492c4ec981a6bbb5f5fbe188/roles/483f28cfc7444513a119d5d363f64518` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:12.926907 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b367e61d-f0bc-44b9-aeef-b5997d17aa0c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e356ef9e492c4ec981a6bbb5f5fbe188, role_id=483f28cfc7444513a119d5d363f64518)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:12.928354 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b367e61d-f0bc-44b9-aeef-b5997d17aa0c None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:12.942366 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 165/331] 10.222.0.15 () {58 vars in 1334 bytes} [Wed Jul 20 14:34:12 2022] PUT /identity/v3/system/users/e356ef9e492c4ec981a6bbb5f5fbe188/roles/483f28cfc7444513a119d5d363f64518 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:34:12.961600 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a6dee-e678-4883-a550-dd6d42fa3b0f None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:12.996687 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:13.002289 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbba141-e0d1-4d81-850b-5a84ec9d10b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:13.004198 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbba141-e0d1-4d81-850b-5a84ec9d10b8 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:13.004198 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbba141-e0d1-4d81-850b-5a84ec9d10b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:13.004198 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbba141-e0d1-4d81-850b-5a84ec9d10b8 None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:13.004569 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbba141-e0d1-4d81-850b-5a84ec9d10b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:13.004887 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbba141-e0d1-4d81-850b-5a84ec9d10b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:13.006329 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbba141-e0d1-4d81-850b-5a84ec9d10b8 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:13.017616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 167/332] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jul 20 14:34:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:13.023939 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367e61d-f0bc-44b9-aeef-b5997d17aa0c None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:13.165610 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367e61d-f0bc-44b9-aeef-b5997d17aa0c None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:13.212560 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bfc6b2-abcb-4b54-8257-d87f600ec049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:13.213481 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfc6b2-abcb-4b54-8257-d87f600ec049 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:13.213761 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfc6b2-abcb-4b54-8257-d87f600ec049 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:13.214031 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfc6b2-abcb-4b54-8257-d87f600ec049 None admin] PATH_INFO: `/v3/users/d533bae8078a47a2b75ceeed7f852760` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:13.215413 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bfc6b2-abcb-4b54-8257-d87f600ec049 None admin] RBAC: Authorizing `identity:get_user(user_id=d533bae8078a47a2b75ceeed7f852760)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:13.216909 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bfc6b2-abcb-4b54-8257-d87f600ec049 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:13.220034 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 166/333] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:34:13 2022] GET /identity/v3/users/d533bae8078a47a2b75ceeed7f852760 => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:13.228266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbba141-e0d1-4d81-850b-5a84ec9d10b8 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:13.375631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbba141-e0d1-4d81-850b-5a84ec9d10b8 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:13.423193 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5874b69-4d2e-4ead-8f69-530c08c0aa9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:13.424235 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5874b69-4d2e-4ead-8f69-530c08c0aa9c None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:13.424523 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5874b69-4d2e-4ead-8f69-530c08c0aa9c None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:13.424804 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5874b69-4d2e-4ead-8f69-530c08c0aa9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:13.425662 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5874b69-4d2e-4ead-8f69-530c08c0aa9c None admin] RBAC: Adding query filter params (role.id=7e394dd183b147ffa15b617f076bbb76, user.id=d533bae8078a47a2b75ceeed7f852760, scope.system=all) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:13.425982 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5874b69-4d2e-4ead-8f69-530c08c0aa9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:13.427492 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5874b69-4d2e-4ead-8f69-530c08c0aa9c None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:13.445709 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 168/334] 10.222.0.15 () {56 vars in 1308 bytes} [Wed Jul 20 14:34:13 2022] GET /identity/v3/role_assignments?user.id=d533bae8078a47a2b75ceeed7f852760&scope.system=all&role.id=7e394dd183b147ffa15b617f076bbb76 => generated 510 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:14.810268 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ea98d-2c16-478f-8457-9b0d7ec91c1f None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:14.810786 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ea98d-2c16-478f-8457-9b0d7ec91c1f None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:14.810860 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ea98d-2c16-478f-8457-9b0d7ec91c1f None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:14.812549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 167/335] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:14.832843 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94c94e-fd23-4cf7-8eb5-6d44c4f9d9e9 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:14.833352 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94c94e-fd23-4cf7-8eb5-6d44c4f9d9e9 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:14.833424 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94c94e-fd23-4cf7-8eb5-6d44c4f9d9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:14.854737 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-6f94c94e-fd23-4cf7-8eb5-6d44c4f9d9e9 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:14.856122 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-6f94c94e-fd23-4cf7-8eb5-6d44c4f9d9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:14.926885 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:14.929145 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ca6b6-802e-4c08-b9d2-851ded40d72a None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:14.929470 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ca6b6-802e-4c08-b9d2-851ded40d72a None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:14.929835 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ca6b6-802e-4c08-b9d2-851ded40d72a None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:14.934278 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 168/336] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:14.941601 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd5420-54df-4fbf-97f1-9973ac0ca187 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:14.942004 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd5420-54df-4fbf-97f1-9973ac0ca187 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:14.942189 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd5420-54df-4fbf-97f1-9973ac0ca187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:14.954426 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 169/337] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:14.962772 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-8cbd5420-54df-4fbf-97f1-9973ac0ca187 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:14.964642 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-8cbd5420-54df-4fbf-97f1-9973ac0ca187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:14.979221 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f94c94e-fd23-4cf7-8eb5-6d44c4f9d9e9 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:14.997534 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:15.052395 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 169/338] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 14:34:15.093501 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f94c94e-fd23-4cf7-8eb5-6d44c4f9d9e9 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:15.097142 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd5420-54df-4fbf-97f1-9973ac0ca187 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:15.139040 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2932de9-c9f8-4bc1-87b0-32d810264ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:15.140322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2932de9-c9f8-4bc1-87b0-32d810264ae3 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:15.140813 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2932de9-c9f8-4bc1-87b0-32d810264ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:15.141286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2932de9-c9f8-4bc1-87b0-32d810264ae3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:15.161647 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2932de9-c9f8-4bc1-87b0-32d810264ae3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:15.171456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-b2932de9-c9f8-4bc1-87b0-32d810264ae3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:34:15.171456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:15.171456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:15.171456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:15.171456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:15.171456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:15.171456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:15.171456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:15.171456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:15.171456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:15.171456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:15.171456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:15.171456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:15.171456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:15.171456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:15.171456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:15.172407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:15.172407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:15.172407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:15.172407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:15.172407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:15.172407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:15.172407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:15.172407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:15.172407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:15.172407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:15.172407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:15.172407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:15.172407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:15.172407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:15.172407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:15.172407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:15.173208 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:15.173208 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:15.173208 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:15.173208 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:15.173208 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:15.173208 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:15.173208 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:15.173208 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:15.173208 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:15.173208 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:15.173208 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:34:15.173208 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:15.187204 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 170/339] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:34:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:34:15.188176 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2932de9-c9f8-4bc1-87b0-32d810264ae3 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:15.299437 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:15.328320 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd5420-54df-4fbf-97f1-9973ac0ca187 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:15.401086 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eef2861-b4e7-4033-a449-86a5fe68bd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:15.403316 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef2861-b4e7-4033-a449-86a5fe68bd7c None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:15.404085 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef2861-b4e7-4033-a449-86a5fe68bd7c None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:15.404896 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef2861-b4e7-4033-a449-86a5fe68bd7c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:15.431161 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eef2861-b4e7-4033-a449-86a5fe68bd7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:15.441041 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2932de9-c9f8-4bc1-87b0-32d810264ae3 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:15.454932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-3eef2861-b4e7-4033-a449-86a5fe68bd7c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 14:34:15.454932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:15.454932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:15.454932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:15.454932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:15.454932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:15.454932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:15.454932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:15.454932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:15.454932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:15.454932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:15.454932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:15.454932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:15.454932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:15.454932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:15.454932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:15.456816 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:15.456816 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:15.456816 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:15.456816 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:15.456816 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:15.456816 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:15.456816 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:15.456816 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:15.456816 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:15.456816 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:15.456816 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:15.456816 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:15.456816 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:15.456816 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:15.456816 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:15.456816 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:15.458272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:15.458272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:15.458272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:15.458272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:15.458272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:15.458272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:15.458272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:15.458272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:15.458272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:15.458272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:15.458272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 14:34:15.458272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:15.468106 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 170/340] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jul 20 14:34:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:34:15.469902 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eef2861-b4e7-4033-a449-86a5fe68bd7c None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:15.518161 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:15.524671 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bfa1f0-cfa7-45dc-b776-9471c2c066f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:15.525362 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bfa1f0-cfa7-45dc-b776-9471c2c066f6 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:15.525642 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bfa1f0-cfa7-45dc-b776-9471c2c066f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:15.525918 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bfa1f0-cfa7-45dc-b776-9471c2c066f6 None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:15.526841 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bfa1f0-cfa7-45dc-b776-9471c2c066f6 None admin] RBAC: Adding query filter params (name=member) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:15.527206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bfa1f0-cfa7-45dc-b776-9471c2c066f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:15.528609 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bfa1f0-cfa7-45dc-b776-9471c2c066f6 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:15.542748 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 171/341] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jul 20 14:34:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:15.544135 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6435c-b286-4c7c-9ba3-905baf3fd51a None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:15.544424 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6435c-b286-4c7c-9ba3-905baf3fd51a None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:15.544702 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6435c-b286-4c7c-9ba3-905baf3fd51a None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:15.545946 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 172/342] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:15.555883 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6435c-b286-4c7c-9ba3-905baf3fd51a None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:15.654167 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eef2861-b4e7-4033-a449-86a5fe68bd7c None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:15.695813 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e6f899-f504-4a8d-a6e7-d74abafeb97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:15.697015 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6f899-f504-4a8d-a6e7-d74abafeb97f None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:15.697509 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6f899-f504-4a8d-a6e7-d74abafeb97f None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:15.698013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6f899-f504-4a8d-a6e7-d74abafeb97f None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:15.699131 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e6f899-f504-4a8d-a6e7-d74abafeb97f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:15.699680 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e6f899-f504-4a8d-a6e7-d74abafeb97f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:15.701420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e6f899-f504-4a8d-a6e7-d74abafeb97f None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:15.711590 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 171/343] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:34:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:15.713487 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabb533-b3ed-4d5b-99b8-5b6691f76ca3 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:15.714024 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabb533-b3ed-4d5b-99b8-5b6691f76ca3 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:15.714508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabb533-b3ed-4d5b-99b8-5b6691f76ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:15.733301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-ebabb533-b3ed-4d5b-99b8-5b6691f76ca3 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:15.734949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-ebabb533-b3ed-4d5b-99b8-5b6691f76ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:15.782443 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6435c-b286-4c7c-9ba3-905baf3fd51a None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:15.786391 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:15.812300 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 172/344] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:15.813884 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebabb533-b3ed-4d5b-99b8-5b6691f76ca3 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:15.826686 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:15.849311 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ebd4c-397a-488c-baf6-eacfc956f224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:15.850593 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ebd4c-397a-488c-baf6-eacfc956f224 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:15.851069 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ebd4c-397a-488c-baf6-eacfc956f224 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:15.851486 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ebd4c-397a-488c-baf6-eacfc956f224 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:15.929577 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ebd4c-397a-488c-baf6-eacfc956f224 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:15.931396 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ebd4c-397a-488c-baf6-eacfc956f224 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:15.991063 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebabb533-b3ed-4d5b-99b8-5b6691f76ca3 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:16.045328 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 173/345] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:34:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:34:16.046386 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39282470-fd28-41a5-a9e3-8690a742a781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:16.046601 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ebd4c-397a-488c-baf6-eacfc956f224 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:16.047201 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39282470-fd28-41a5-a9e3-8690a742a781 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:16.047382 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39282470-fd28-41a5-a9e3-8690a742a781 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:16.047664 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39282470-fd28-41a5-a9e3-8690a742a781 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:16.051272 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:16.166854 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39282470-fd28-41a5-a9e3-8690a742a781 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:16.169191 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39282470-fd28-41a5-a9e3-8690a742a781 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:16.187415 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:16.205382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ebd4c-397a-488c-baf6-eacfc956f224 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:16.261302 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d57abe-6683-414b-8d01-27476e6be98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:16.262368 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d57abe-6683-414b-8d01-27476e6be98c None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:16.262479 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d57abe-6683-414b-8d01-27476e6be98c None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:16.262715 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d57abe-6683-414b-8d01-27476e6be98c None admin] PATH_INFO: `/v3/roles/member` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:16.277356 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d57abe-6683-414b-8d01-27476e6be98c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:16.289598 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-24d57abe-6683-414b-8d01-27476e6be98c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:34:16.289598 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:16.289598 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:16.289598 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:16.289598 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:16.289598 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:16.289598 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:16.289598 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:16.289598 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:16.289598 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:16.289598 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:16.289598 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:16.289598 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:16.289598 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:16.289598 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:16.289598 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:16.290631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:16.290631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:16.290631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:16.290631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:16.290631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:16.290631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:16.290631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:16.290631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:16.290631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:16.290631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:16.290631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:16.290631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:16.290631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:16.290631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:16.290631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:16.290631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:16.291628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:16.291628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:16.291628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:16.291628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:16.291628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:16.291628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:16.291628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:16.291628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:16.291628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:16.291628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:16.291628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:34:16.291628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:16.291628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 174/346] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:34:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:16.292320 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d57abe-6683-414b-8d01-27476e6be98c None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:16.322676 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 173/347] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:34:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:34:16.324639 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255711a-6353-4bf8-9943-010500700e9e None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:16.325145 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255711a-6353-4bf8-9943-010500700e9e None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:16.325649 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255711a-6353-4bf8-9943-010500700e9e None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:16.327704 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 174/348] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:16.329021 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255711a-6353-4bf8-9943-010500700e9e None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:16.386205 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:16.448740 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d57abe-6683-414b-8d01-27476e6be98c None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:16.464662 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255711a-6353-4bf8-9943-010500700e9e None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:16.477451 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:16.495880 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb97ad-1de9-46b0-a518-393495231b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:16.496807 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb97ad-1de9-46b0-a518-393495231b55 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:16.497334 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb97ad-1de9-46b0-a518-393495231b55 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:16.497619 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb97ad-1de9-46b0-a518-393495231b55 None admin] PATH_INFO: `/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:16.499586 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb97ad-1de9-46b0-a518-393495231b55 None admin] RBAC: Authorizing `identity:get_project(project_id=e27e1edc913f4b99a1b0be6ab9d16b6f)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:16.501744 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb97ad-1de9-46b0-a518-393495231b55 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:16.504361 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 175/349] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:34:16 2022] GET /identity/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:16.505705 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb97ad-1de9-46b0-a518-393495231b55 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:16.510576 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3bb6a-22fb-42e0-b5de-f14390c07b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:16.512867 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3bb6a-22fb-42e0-b5de-f14390c07b53 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:16.513354 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3bb6a-22fb-42e0-b5de-f14390c07b53 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:16.513756 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3bb6a-22fb-42e0-b5de-f14390c07b53 None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:16.514653 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb3bb6a-22fb-42e0-b5de-f14390c07b53 None admin] RBAC: Adding query filter params (name=member) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:16.515192 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb3bb6a-22fb-42e0-b5de-f14390c07b53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:16.516890 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb3bb6a-22fb-42e0-b5de-f14390c07b53 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:16.533379 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 175/350] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jul 20 14:34:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:16.535332 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9186150-b19c-4883-93e7-2cfca15f992d None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:16.535806 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9186150-b19c-4883-93e7-2cfca15f992d None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:16.536250 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9186150-b19c-4883-93e7-2cfca15f992d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:16.559552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-b9186150-b19c-4883-93e7-2cfca15f992d None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:16.560764 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-b9186150-b19c-4883-93e7-2cfca15f992d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:16.633334 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb97ad-1de9-46b0-a518-393495231b55 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:16.648481 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 176/351] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:16.649520 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9186150-b19c-4883-93e7-2cfca15f992d None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:16.675218 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7560a4-d770-4322-bdda-37619e6f8ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:16.676204 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7560a4-d770-4322-bdda-37619e6f8ef7 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:16.676507 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7560a4-d770-4322-bdda-37619e6f8ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:16.676795 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7560a4-d770-4322-bdda-37619e6f8ef7 None admin] PATH_INFO: `/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:16.678040 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7560a4-d770-4322-bdda-37619e6f8ef7 None admin] RBAC: Authorizing `identity:get_project(project_id=e27e1edc913f4b99a1b0be6ab9d16b6f)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:16.678529 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:16.681536 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7560a4-d770-4322-bdda-37619e6f8ef7 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:16.684064 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 176/352] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:34:16 2022] GET /identity/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:16.685098 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7560a4-d770-4322-bdda-37619e6f8ef7 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:16.794927 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9186150-b19c-4883-93e7-2cfca15f992d None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:16.796704 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:16.812988 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7560a4-d770-4322-bdda-37619e6f8ef7 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:16.843453 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f94848-702e-4e02-9b4e-0bf5f5f416ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:16.844334 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f94848-702e-4e02-9b4e-0bf5f5f416ba None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:16.844661 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f94848-702e-4e02-9b4e-0bf5f5f416ba None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:16.844879 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f94848-702e-4e02-9b4e-0bf5f5f416ba None admin] PATH_INFO: `/v3/groups/fc8ddab7364d4c7ba121fe83f01187c2` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:16.851267 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f94848-702e-4e02-9b4e-0bf5f5f416ba None admin] RBAC: Authorizing `identity:get_group(group_id=fc8ddab7364d4c7ba121fe83f01187c2)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:16.852736 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f94848-702e-4e02-9b4e-0bf5f5f416ba None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:16.853780 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00224f-4497-419e-9931-aea2c16ba95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:16.854481 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00224f-4497-419e-9931-aea2c16ba95a None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:16.854770 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00224f-4497-419e-9931-aea2c16ba95a None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:16.855072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00224f-4497-419e-9931-aea2c16ba95a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:16.855766 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 177/353] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Jul 20 14:34:16 2022] GET /identity/v3/groups/fc8ddab7364d4c7ba121fe83f01187c2 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:16.857173 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992dc402-21e6-4bfa-8a35-9c6f59f87802 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:16.857463 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992dc402-21e6-4bfa-8a35-9c6f59f87802 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:16.857735 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992dc402-21e6-4bfa-8a35-9c6f59f87802 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:16.858993 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 178/354] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:16.859785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992dc402-21e6-4bfa-8a35-9c6f59f87802 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:16.932074 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:16.935508 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00224f-4497-419e-9931-aea2c16ba95a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:16.938674 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00224f-4497-419e-9931-aea2c16ba95a None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:16.984504 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992dc402-21e6-4bfa-8a35-9c6f59f87802 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:17.031300 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50e194-bae2-4f91-a9e2-55e2c48a0370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:17.032185 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50e194-bae2-4f91-a9e2-55e2c48a0370 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:17.032473 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50e194-bae2-4f91-a9e2-55e2c48a0370 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:17.032748 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50e194-bae2-4f91-a9e2-55e2c48a0370 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:17.047991 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe50e194-bae2-4f91-a9e2-55e2c48a0370 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:17.060094 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-fe50e194-bae2-4f91-a9e2-55e2c48a0370 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 14:34:17.060094 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:17.060094 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:17.060094 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:17.060094 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:17.060094 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:17.060094 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:17.060094 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:17.060094 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:17.060094 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:17.060094 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:17.060094 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:17.060094 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:17.060094 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:17.060094 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:17.060094 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:17.061039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:17.061039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:17.061039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:17.061039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:17.061039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:17.061039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:17.061039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:17.061039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:17.061039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:17.061039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:17.061039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:17.061039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:17.061039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:17.061039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:17.061039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:17.061039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:17.062052 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:17.062052 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:17.062052 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:17.062052 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:17.062052 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:17.062052 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:17.062052 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:17.062052 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:17.062052 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:17.062052 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:17.062052 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 14:34:17.062052 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:17.062052 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 179/355] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jul 20 14:34:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:17.062821 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50e194-bae2-4f91-a9e2-55e2c48a0370 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:17.069125 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 177/356] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:34:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:34:17.070648 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b898c-8539-4ded-9170-8aaaa0f18c3e None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:17.070940 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b898c-8539-4ded-9170-8aaaa0f18c3e None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:17.071279 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b898c-8539-4ded-9170-8aaaa0f18c3e None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:17.072908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 178/357] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:17.073563 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b898c-8539-4ded-9170-8aaaa0f18c3e None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:17.108860 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:17.196151 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50e194-bae2-4f91-a9e2-55e2c48a0370 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:17.208308 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:17.214106 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b898c-8539-4ded-9170-8aaaa0f18c3e None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:17.236712 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c32ad-44eb-4eed-b58e-b588490c96d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:17.237852 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c32ad-44eb-4eed-b58e-b588490c96d8 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:17.238308 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c32ad-44eb-4eed-b58e-b588490c96d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:17.238581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c32ad-44eb-4eed-b58e-b588490c96d8 None admin] PATH_INFO: `/v3/groups/fc8ddab7364d4c7ba121fe83f01187c2` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:17.239846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392c32ad-44eb-4eed-b58e-b588490c96d8 None admin] RBAC: Authorizing `identity:get_group(group_id=fc8ddab7364d4c7ba121fe83f01187c2)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:17.242092 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392c32ad-44eb-4eed-b58e-b588490c96d8 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:17.244114 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 180/358] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Jul 20 14:34:17 2022] GET /identity/v3/groups/fc8ddab7364d4c7ba121fe83f01187c2 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:17.245341 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca096da-5311-4cf0-a4bd-579eedfcb951 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:17.245685 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca096da-5311-4cf0-a4bd-579eedfcb951 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:17.245961 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca096da-5311-4cf0-a4bd-579eedfcb951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:17.252141 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ebbe07-3006-4570-9431-a6a8bab445cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:17.252817 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebbe07-3006-4570-9431-a6a8bab445cc None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:17.253107 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebbe07-3006-4570-9431-a6a8bab445cc None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:17.253418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebbe07-3006-4570-9431-a6a8bab445cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:17.254247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ebbe07-3006-4570-9431-a6a8bab445cc None admin] RBAC: Adding query filter params (group.id=fc8ddab7364d4c7ba121fe83f01187c2, role.id=7e394dd183b147ffa15b617f076bbb76, scope.project.id=e27e1edc913f4b99a1b0be6ab9d16b6f) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:17.254560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ebbe07-3006-4570-9431-a6a8bab445cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:17.256112 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ebbe07-3006-4570-9431-a6a8bab445cc None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:17.264749 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 179/359] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Jul 20 14:34:17 2022] GET /identity/v3/role_assignments?group.id=fc8ddab7364d4c7ba121fe83f01187c2&scope.project.id=e27e1edc913f4b99a1b0be6ab9d16b6f&role.id=7e394dd183b147ffa15b617f076bbb76 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:17.264927 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-6ca096da-5311-4cf0-a4bd-579eedfcb951 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:17.265024 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ebbe07-3006-4570-9431-a6a8bab445cc None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:17.271130 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-6ca096da-5311-4cf0-a4bd-579eedfcb951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:17.361129 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 181/360] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:17.362004 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca096da-5311-4cf0-a4bd-579eedfcb951 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:17.393881 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:17.410131 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ebbe07-3006-4570-9431-a6a8bab445cc None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:17.454237 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb046937-2780-4373-88c9-13be5964600c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:17.454934 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb046937-2780-4373-88c9-13be5964600c None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:17.455277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb046937-2780-4373-88c9-13be5964600c None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:17.455516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb046937-2780-4373-88c9-13be5964600c None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:17.456389 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb046937-2780-4373-88c9-13be5964600c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:17.456690 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb046937-2780-4373-88c9-13be5964600c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:17.458093 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb046937-2780-4373-88c9-13be5964600c None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:17.472584 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 180/361] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:34:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:17.473972 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66576a6-b7ad-4696-88c4-7e2c14b24fed None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:17.474266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66576a6-b7ad-4696-88c4-7e2c14b24fed None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:17.474545 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66576a6-b7ad-4696-88c4-7e2c14b24fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:17.500777 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca096da-5311-4cf0-a4bd-579eedfcb951 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:17.504797 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-f66576a6-b7ad-4696-88c4-7e2c14b24fed None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:17.506184 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-f66576a6-b7ad-4696-88c4-7e2c14b24fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:17.534100 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:17.549440 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3997b55-57aa-47a4-a5e2-18d51bb38a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:17.550329 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3997b55-57aa-47a4-a5e2-18d51bb38a9f None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:17.550721 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3997b55-57aa-47a4-a5e2-18d51bb38a9f None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:17.551105 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3997b55-57aa-47a4-a5e2-18d51bb38a9f None admin] PATH_INFO: `/v3/projects/888e6ef050b643db90e7280d61f3e3e8` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:17.552250 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3997b55-57aa-47a4-a5e2-18d51bb38a9f None admin] RBAC: Authorizing `identity:get_project(project_id=888e6ef050b643db90e7280d61f3e3e8)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:17.553781 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3997b55-57aa-47a4-a5e2-18d51bb38a9f None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:17.555700 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 182/362] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:34:17 2022] GET /identity/v3/projects/888e6ef050b643db90e7280d61f3e3e8 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:17.556468 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3997b55-57aa-47a4-a5e2-18d51bb38a9f None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:17.595012 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 181/363] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:17.595998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66576a6-b7ad-4696-88c4-7e2c14b24fed None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:17.677260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:17.686677 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3997b55-57aa-47a4-a5e2-18d51bb38a9f None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:17.719641 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66576a6-b7ad-4696-88c4-7e2c14b24fed None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:17.735973 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1b692-2ece-469e-a7b8-bc4466f55e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:17.736672 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1b692-2ece-469e-a7b8-bc4466f55e7a None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:17.736923 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1b692-2ece-469e-a7b8-bc4466f55e7a None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:17.737201 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1b692-2ece-469e-a7b8-bc4466f55e7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:17.738053 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d1b692-2ece-469e-a7b8-bc4466f55e7a None admin] RBAC: Adding query filter params (group.id=fc8ddab7364d4c7ba121fe83f01187c2, role.id=b74189174ce24c1382712858dc35f3ce, scope.project.id=e27e1edc913f4b99a1b0be6ab9d16b6f) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:17.738368 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d1b692-2ece-469e-a7b8-bc4466f55e7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:17.739938 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d1b692-2ece-469e-a7b8-bc4466f55e7a None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:17.746517 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 183/364] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Jul 20 14:34:17 2022] GET /identity/v3/role_assignments?group.id=fc8ddab7364d4c7ba121fe83f01187c2&scope.project.id=e27e1edc913f4b99a1b0be6ab9d16b6f&role.id=b74189174ce24c1382712858dc35f3ce => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:17.747221 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1b692-2ece-469e-a7b8-bc4466f55e7a None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:17.770265 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd393027-3803-4d7d-857d-4034e819d4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:17.771132 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd393027-3803-4d7d-857d-4034e819d4a6 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:17.771516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd393027-3803-4d7d-857d-4034e819d4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:17.771803 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd393027-3803-4d7d-857d-4034e819d4a6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:17.786257 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd393027-3803-4d7d-857d-4034e819d4a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:17.795263 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-fd393027-3803-4d7d-857d-4034e819d4a6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 14:34:17.795263 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:17.795263 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:17.795263 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:17.795263 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:17.795263 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:17.795263 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:17.795263 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:17.795263 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:17.795263 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:17.795263 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:17.795263 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:17.795263 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:17.795263 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:17.795263 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:17.795263 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:17.796276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:17.796276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:17.796276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:17.796276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:17.796276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:17.796276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:17.796276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:17.796276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:17.796276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:17.796276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:17.796276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:17.796276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:17.796276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:17.796276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:17.796276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:17.796276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:17.797548 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:17.797548 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:17.797548 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:17.797548 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:17.797548 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:17.797548 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:17.797548 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:17.797548 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:17.797548 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:17.797548 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:17.797548 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 14:34:17.797548 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:17.797548 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 182/365] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:34:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:17.798262 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd393027-3803-4d7d-857d-4034e819d4a6 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:17.820639 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:17.872629 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1b692-2ece-469e-a7b8-bc4466f55e7a None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:17.914453 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e730ff-5966-47cc-990d-038481b7b246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:17.915171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e730ff-5966-47cc-990d-038481b7b246 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:17.915457 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e730ff-5966-47cc-990d-038481b7b246 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:17.915736 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e730ff-5966-47cc-990d-038481b7b246 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:17.933055 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd393027-3803-4d7d-857d-4034e819d4a6 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:17.946380 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:17.983751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7f89ba-db11-476b-8b3e-c83f83ad0a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:17.985153 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f89ba-db11-476b-8b3e-c83f83ad0a2b None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:17.985661 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f89ba-db11-476b-8b3e-c83f83ad0a2b None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:17.986008 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f89ba-db11-476b-8b3e-c83f83ad0a2b None admin] PATH_INFO: `/v3/groups/fc8ddab7364d4c7ba121fe83f01187c2` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:17.987419 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7f89ba-db11-476b-8b3e-c83f83ad0a2b None admin] RBAC: Authorizing `identity:get_group(group_id=fc8ddab7364d4c7ba121fe83f01187c2)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:17.989024 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7f89ba-db11-476b-8b3e-c83f83ad0a2b None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:17.991206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 183/366] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Jul 20 14:34:17 2022] GET /identity/v3/groups/fc8ddab7364d4c7ba121fe83f01187c2 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:17.992096 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7f89ba-db11-476b-8b3e-c83f83ad0a2b None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:17.998734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e730ff-5966-47cc-990d-038481b7b246 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:18.000291 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e730ff-5966-47cc-990d-038481b7b246 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:18.097316 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:18.117563 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7f89ba-db11-476b-8b3e-c83f83ad0a2b None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:18.150581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 184/367] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:34:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:34:18.151846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e730ff-5966-47cc-990d-038481b7b246 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:18.163859 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ec7ee-58b1-4be1-b0d1-d5e5312fefc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:18.164820 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ec7ee-58b1-4be1-b0d1-d5e5312fefc9 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:18.165171 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ec7ee-58b1-4be1-b0d1-d5e5312fefc9 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:18.165453 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ec7ee-58b1-4be1-b0d1-d5e5312fefc9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:18.178041 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846ec7ee-58b1-4be1-b0d1-d5e5312fefc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:18.185937 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-846ec7ee-58b1-4be1-b0d1-d5e5312fefc9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:34:18.185937 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:18.185937 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:18.185937 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:18.185937 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:18.185937 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:18.185937 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:18.185937 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:18.185937 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:18.185937 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:18.185937 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:18.185937 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:18.185937 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:18.185937 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:18.185937 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:18.185937 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:18.186848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:18.186848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:18.186848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:18.186848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:18.186848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:18.186848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:18.186848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:18.186848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:18.186848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:18.186848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:18.186848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:18.186848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:18.186848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:18.186848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:18.186848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:18.186848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:18.187722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:18.187722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:18.187722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:18.187722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:18.187722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:18.187722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:18.187722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:18.187722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:18.187722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:18.187722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:18.187722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:34:18.187722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:18.188887 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 184/368] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:34:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:18.190120 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ec7ee-58b1-4be1-b0d1-d5e5312fefc9 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:18.222702 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:18.286173 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e730ff-5966-47cc-990d-038481b7b246 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:18.318928 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ec7ee-58b1-4be1-b0d1-d5e5312fefc9 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:18.328090 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a530dd3-1b00-4e2d-b8a3-5b36aebfde62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:18.329080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a530dd3-1b00-4e2d-b8a3-5b36aebfde62 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:18.329552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a530dd3-1b00-4e2d-b8a3-5b36aebfde62 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:18.330323 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a530dd3-1b00-4e2d-b8a3-5b36aebfde62 None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:18.331293 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a530dd3-1b00-4e2d-b8a3-5b36aebfde62 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:18.331795 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a530dd3-1b00-4e2d-b8a3-5b36aebfde62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:18.333675 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a530dd3-1b00-4e2d-b8a3-5b36aebfde62 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:18.344705 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 185/369] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jul 20 14:34:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:18.346308 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a530dd3-1b00-4e2d-b8a3-5b36aebfde62 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:18.369967 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e3f83-a7a7-49c8-982e-8a88e1dd2149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:18.370713 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e3f83-a7a7-49c8-982e-8a88e1dd2149 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:18.371003 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e3f83-a7a7-49c8-982e-8a88e1dd2149 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:18.371322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e3f83-a7a7-49c8-982e-8a88e1dd2149 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:18.372288 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3e3f83-a7a7-49c8-982e-8a88e1dd2149 None admin] RBAC: Adding query filter params (group.id=fc8ddab7364d4c7ba121fe83f01187c2, role.id=7e394dd183b147ffa15b617f076bbb76, scope.project.id=888e6ef050b643db90e7280d61f3e3e8) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:18.372611 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3e3f83-a7a7-49c8-982e-8a88e1dd2149 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:18.374190 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3e3f83-a7a7-49c8-982e-8a88e1dd2149 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:18.380638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 185/370] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Jul 20 14:34:18 2022] GET /identity/v3/role_assignments?group.id=fc8ddab7364d4c7ba121fe83f01187c2&scope.project.id=888e6ef050b643db90e7280d61f3e3e8&role.id=7e394dd183b147ffa15b617f076bbb76 => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:18.381463 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e3f83-a7a7-49c8-982e-8a88e1dd2149 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:18.400473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:18.473446 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a530dd3-1b00-4e2d-b8a3-5b36aebfde62 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:18.503518 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:18.512109 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e3f83-a7a7-49c8-982e-8a88e1dd2149 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:18.514056 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87521364-9b01-4f46-a173-8473238d20ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:18.514998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87521364-9b01-4f46-a173-8473238d20ed None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:18.515315 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87521364-9b01-4f46-a173-8473238d20ed None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:18.515604 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87521364-9b01-4f46-a173-8473238d20ed None admin] PATH_INFO: `/v3/projects/888e6ef050b643db90e7280d61f3e3e8` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:18.516978 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87521364-9b01-4f46-a173-8473238d20ed None admin] RBAC: Authorizing `identity:get_project(project_id=888e6ef050b643db90e7280d61f3e3e8)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:18.518527 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87521364-9b01-4f46-a173-8473238d20ed None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:18.520592 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 186/371] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:34:18 2022] GET /identity/v3/projects/888e6ef050b643db90e7280d61f3e3e8 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:18.521282 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87521364-9b01-4f46-a173-8473238d20ed None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:18.561512 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa3d77-9b15-4a52-ab36-261554eea1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:18.562325 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa3d77-9b15-4a52-ab36-261554eea1b9 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:18.562609 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa3d77-9b15-4a52-ab36-261554eea1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:18.562896 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa3d77-9b15-4a52-ab36-261554eea1b9 None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:18.563998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaa3d77-9b15-4a52-ab36-261554eea1b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:18.564338 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaa3d77-9b15-4a52-ab36-261554eea1b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:18.565901 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaa3d77-9b15-4a52-ab36-261554eea1b9 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:18.576612 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 186/372] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:34:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:18.577632 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa3d77-9b15-4a52-ab36-261554eea1b9 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:18.661906 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87521364-9b01-4f46-a173-8473238d20ed None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:18.680819 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:18.706186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f80bf-58c5-47cc-af47-795ca7c9affe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:18.707273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f80bf-58c5-47cc-af47-795ca7c9affe None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:18.707565 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f80bf-58c5-47cc-af47-795ca7c9affe None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:18.707841 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f80bf-58c5-47cc-af47-795ca7c9affe None admin] PATH_INFO: `/v3/users/e356ef9e492c4ec981a6bbb5f5fbe188` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:18.709954 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8f80bf-58c5-47cc-af47-795ca7c9affe None admin] RBAC: Authorizing `identity:get_user(user_id=e356ef9e492c4ec981a6bbb5f5fbe188)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:18.711692 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8f80bf-58c5-47cc-af47-795ca7c9affe None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:18.713732 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 187/373] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:34:18 2022] GET /identity/v3/users/e356ef9e492c4ec981a6bbb5f5fbe188 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:18.717428 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f80bf-58c5-47cc-af47-795ca7c9affe None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:18.743113 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa3d77-9b15-4a52-ab36-261554eea1b9 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:18.803730 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:18.816498 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a09f08-7726-42ae-9b09-f5f42af6c5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:18.817439 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a09f08-7726-42ae-9b09-f5f42af6c5b0 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:18.817726 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a09f08-7726-42ae-9b09-f5f42af6c5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:18.818009 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a09f08-7726-42ae-9b09-f5f42af6c5b0 None admin] PATH_INFO: `/v3/groups/fc8ddab7364d4c7ba121fe83f01187c2` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:18.819466 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a09f08-7726-42ae-9b09-f5f42af6c5b0 None admin] RBAC: Authorizing `identity:get_group(group_id=fc8ddab7364d4c7ba121fe83f01187c2)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:18.821100 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a09f08-7726-42ae-9b09-f5f42af6c5b0 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:18.823118 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 187/374] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Jul 20 14:34:18 2022] GET /identity/v3/groups/fc8ddab7364d4c7ba121fe83f01187c2 => generated 226 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:18.823771 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a09f08-7726-42ae-9b09-f5f42af6c5b0 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:18.908496 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f80bf-58c5-47cc-af47-795ca7c9affe None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:18.966876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a09f08-7726-42ae-9b09-f5f42af6c5b0 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:18.981791 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ab18f-0282-4922-9e3d-7c31c16f3f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:18.982955 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ab18f-0282-4922-9e3d-7c31c16f3f42 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:18.983537 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ab18f-0282-4922-9e3d-7c31c16f3f42 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:18.984065 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ab18f-0282-4922-9e3d-7c31c16f3f42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:19.007496 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18ab3f-d21e-4fbb-b434-1bd1b7f61f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:19.008412 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18ab3f-d21e-4fbb-b434-1bd1b7f61f69 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:19.008706 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18ab3f-d21e-4fbb-b434-1bd1b7f61f69 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:19.008991 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18ab3f-d21e-4fbb-b434-1bd1b7f61f69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:19.010007 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac18ab3f-d21e-4fbb-b434-1bd1b7f61f69 None admin] RBAC: Adding query filter params (role.id=483f28cfc7444513a119d5d363f64518, user.id=e356ef9e492c4ec981a6bbb5f5fbe188, scope.system=all) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:19.010349 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac18ab3f-d21e-4fbb-b434-1bd1b7f61f69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:19.011949 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac18ab3f-d21e-4fbb-b434-1bd1b7f61f69 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:19.020785 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:19.037406 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 188/375] 10.222.0.15 () {56 vars in 1308 bytes} [Wed Jul 20 14:34:18 2022] GET /identity/v3/role_assignments?user.id=e356ef9e492c4ec981a6bbb5f5fbe188&scope.system=all&role.id=483f28cfc7444513a119d5d363f64518 => generated 510 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:19.038408 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18ab3f-d21e-4fbb-b434-1bd1b7f61f69 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:19.146748 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ab18f-0282-4922-9e3d-7c31c16f3f42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:19.148447 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ab18f-0282-4922-9e3d-7c31c16f3f42 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:19.269481 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:19.325382 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 188/376] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:34:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:34:19.334795 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ab18f-0282-4922-9e3d-7c31c16f3f42 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:19.335382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18ab3f-d21e-4fbb-b434-1bd1b7f61f69 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:19.412256 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005fbd22-09c4-4d2f-96e8-ea540e2437a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:19.412959 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fbd22-09c4-4d2f-96e8-ea540e2437a6 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:19.413244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fbd22-09c4-4d2f-96e8-ea540e2437a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:19.413584 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fbd22-09c4-4d2f-96e8-ea540e2437a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:19.414731 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005fbd22-09c4-4d2f-96e8-ea540e2437a6 None admin] RBAC: Adding query filter params (group.id=fc8ddab7364d4c7ba121fe83f01187c2, role.id=b74189174ce24c1382712858dc35f3ce, scope.project.id=888e6ef050b643db90e7280d61f3e3e8) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:19.415067 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005fbd22-09c4-4d2f-96e8-ea540e2437a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:19.416765 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005fbd22-09c4-4d2f-96e8-ea540e2437a6 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:19.424987 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 189/377] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Jul 20 14:34:19 2022] GET /identity/v3/role_assignments?group.id=fc8ddab7364d4c7ba121fe83f01187c2&scope.project.id=888e6ef050b643db90e7280d61f3e3e8&role.id=b74189174ce24c1382712858dc35f3ce => generated 260 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:19.499135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ab18f-0282-4922-9e3d-7c31c16f3f42 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:19.549240 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc322ef6-4ce9-4cdd-9b81-1321f5a2c0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:19.550134 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc322ef6-4ce9-4cdd-9b81-1321f5a2c0f2 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:19.550417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc322ef6-4ce9-4cdd-9b81-1321f5a2c0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:19.550698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc322ef6-4ce9-4cdd-9b81-1321f5a2c0f2 None admin] PATH_INFO: `/v3/projects/3ca83facd1c8420dacb9e91e8abe0fd4` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:19.551926 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc322ef6-4ce9-4cdd-9b81-1321f5a2c0f2 None admin] RBAC: Authorizing `identity:get_project(project_id=3ca83facd1c8420dacb9e91e8abe0fd4)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:19.553579 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc322ef6-4ce9-4cdd-9b81-1321f5a2c0f2 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:19.555756 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 189/378] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:34:19 2022] GET /identity/v3/projects/3ca83facd1c8420dacb9e91e8abe0fd4 => generated 344 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:19.561594 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005fbd22-09c4-4d2f-96e8-ea540e2437a6 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:19.651112 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:19.699282 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005fbd22-09c4-4d2f-96e8-ea540e2437a6 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:19.734018 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36b626-3e8c-473a-a556-3d9aa70b0e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:19.734898 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36b626-3e8c-473a-a556-3d9aa70b0e4f None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:19.735209 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36b626-3e8c-473a-a556-3d9aa70b0e4f None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:19.735484 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36b626-3e8c-473a-a556-3d9aa70b0e4f None admin] PATH_INFO: `/v3/groups/50b15137c74c40df9d5f00a6a168e3cd` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:19.741968 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36b626-3e8c-473a-a556-3d9aa70b0e4f None admin] RBAC: Authorizing `identity:get_group(group_id=50b15137c74c40df9d5f00a6a168e3cd)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:19.743469 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36b626-3e8c-473a-a556-3d9aa70b0e4f None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:19.745502 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 190/379] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Jul 20 14:34:19 2022] GET /identity/v3/groups/50b15137c74c40df9d5f00a6a168e3cd => generated 229 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:19.752426 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc322ef6-4ce9-4cdd-9b81-1321f5a2c0f2 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:19.786230 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:19.862216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88bef65-7c8f-41dd-be44-4f0c8020f152 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:19.862390 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88bef65-7c8f-41dd-be44-4f0c8020f152 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:19.862725 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88bef65-7c8f-41dd-be44-4f0c8020f152 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:19.864845 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 191/380] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 14:34:19.869834 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e51602-51d2-439c-8507-6d2f984b7d8e None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:19.870125 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e51602-51d2-439c-8507-6d2f984b7d8e None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:19.870408 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e51602-51d2-439c-8507-6d2f984b7d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:19.893058 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc322ef6-4ce9-4cdd-9b81-1321f5a2c0f2 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:19.895626 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-c7e51602-51d2-439c-8507-6d2f984b7d8e None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:19.896834 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-c7e51602-51d2-439c-8507-6d2f984b7d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:19.953562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1aef77-0f8a-4aca-a777-aedbb4d6a697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:19.954265 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1aef77-0f8a-4aca-a777-aedbb4d6a697 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:19.954517 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1aef77-0f8a-4aca-a777-aedbb4d6a697 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:19.954796 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1aef77-0f8a-4aca-a777-aedbb4d6a697 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:19.955679 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1aef77-0f8a-4aca-a777-aedbb4d6a697 None admin] RBAC: Adding query filter params (group.id=50b15137c74c40df9d5f00a6a168e3cd, role.id=49bd5abf255048a0abaa9538e309f516, scope.project.id=3ca83facd1c8420dacb9e91e8abe0fd4) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:19.955994 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1aef77-0f8a-4aca-a777-aedbb4d6a697 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:19.957437 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1aef77-0f8a-4aca-a777-aedbb4d6a697 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:19.971736 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 190/381] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Jul 20 14:34:19 2022] GET /identity/v3/role_assignments?group.id=50b15137c74c40df9d5f00a6a168e3cd&scope.project.id=3ca83facd1c8420dacb9e91e8abe0fd4&role.id=49bd5abf255048a0abaa9538e309f516 => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:20.031449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 192/382] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 14:34:20.066876 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1aef77-0f8a-4aca-a777-aedbb4d6a697 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:20.168793 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:20.193491 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1aef77-0f8a-4aca-a777-aedbb4d6a697 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:20.234251 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8eefb-bac0-4909-b733-56b531aa2717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:20.235101 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8eefb-bac0-4909-b733-56b531aa2717 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:20.235390 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8eefb-bac0-4909-b733-56b531aa2717 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:20.235667 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8eefb-bac0-4909-b733-56b531aa2717 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:20.248217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d8eefb-bac0-4909-b733-56b531aa2717 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:20.255986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-77d8eefb-bac0-4909-b733-56b531aa2717 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:34:20.255986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:20.255986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:20.255986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:20.255986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:20.255986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:20.255986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:20.255986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:20.255986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:20.255986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:20.255986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:20.255986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:20.255986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:20.255986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:20.255986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:20.255986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:20.256885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:20.256885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:20.256885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:20.256885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:20.256885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:20.256885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:20.256885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:20.256885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:20.256885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:20.256885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:20.256885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:20.256885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:20.256885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:20.256885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:20.256885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:20.256885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:20.257727 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:20.257727 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:20.257727 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:20.257727 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:20.257727 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:20.257727 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:20.257727 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:20.257727 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:20.257727 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:20.257727 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:20.257727 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:34:20.257727 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:20.262477 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 191/383] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:34:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:20.263866 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e51602-51d2-439c-8507-6d2f984b7d8e None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:20.330051 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:20.384374 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e51602-51d2-439c-8507-6d2f984b7d8e None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:20.426956 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47dc373-7050-40b8-b6ce-8aef50d3d5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:20.427601 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dc373-7050-40b8-b6ce-8aef50d3d5c9 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:20.427889 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dc373-7050-40b8-b6ce-8aef50d3d5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:20.428166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dc373-7050-40b8-b6ce-8aef50d3d5c9 None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:20.428943 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47dc373-7050-40b8-b6ce-8aef50d3d5c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:20.429313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47dc373-7050-40b8-b6ce-8aef50d3d5c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:20.430717 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47dc373-7050-40b8-b6ce-8aef50d3d5c9 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:20.440272 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 193/384] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jul 20 14:34:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:20.446043 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8eefb-bac0-4909-b733-56b531aa2717 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:20.502024 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5dcdfb-aa26-476a-b0c8-ec920f131710 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:20.502357 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5dcdfb-aa26-476a-b0c8-ec920f131710 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:20.502591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5dcdfb-aa26-476a-b0c8-ec920f131710 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:20.504089 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 194/385] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 14:34:20.509319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40d518-3717-43b0-9ce7-564b9e46c417 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:20.509658 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40d518-3717-43b0-9ce7-564b9e46c417 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:20.509959 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40d518-3717-43b0-9ce7-564b9e46c417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:20.541006 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-2b40d518-3717-43b0-9ce7-564b9e46c417 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:20.542462 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-2b40d518-3717-43b0-9ce7-564b9e46c417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:20.638815 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 195/386] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 14:34:20.640960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8eefb-bac0-4909-b733-56b531aa2717 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:20.661409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b40d518-3717-43b0-9ce7-564b9e46c417 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:20.710384 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73135d2-3cf2-4d55-b2ac-3ca0129e7e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:20.711150 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73135d2-3cf2-4d55-b2ac-3ca0129e7e11 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:20.711434 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73135d2-3cf2-4d55-b2ac-3ca0129e7e11 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:20.711708 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73135d2-3cf2-4d55-b2ac-3ca0129e7e11 None admin] PATH_INFO: `/v3/groups/fc8ddab7364d4c7ba121fe83f01187c2` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:20.712884 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73135d2-3cf2-4d55-b2ac-3ca0129e7e11 None admin] RBAC: Authorizing `identity:get_group(group_id=fc8ddab7364d4c7ba121fe83f01187c2)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:20.714354 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73135d2-3cf2-4d55-b2ac-3ca0129e7e11 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:20.715390 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:20.716361 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 192/387] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Jul 20 14:34:20 2022] GET /identity/v3/groups/fc8ddab7364d4c7ba121fe83f01187c2 => generated 226 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:20.724248 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73135d2-3cf2-4d55-b2ac-3ca0129e7e11 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:20.743364 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:20.839200 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73135d2-3cf2-4d55-b2ac-3ca0129e7e11 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:20.844019 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b40d518-3717-43b0-9ce7-564b9e46c417 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:20.881987 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49379ae-1eaf-4b96-99dd-8e0bd0cdc228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:20.882991 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49379ae-1eaf-4b96-99dd-8e0bd0cdc228 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:20.887005 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49379ae-1eaf-4b96-99dd-8e0bd0cdc228 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:20.887320 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49379ae-1eaf-4b96-99dd-8e0bd0cdc228 None admin] PATH_INFO: `/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:20.889629 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49379ae-1eaf-4b96-99dd-8e0bd0cdc228 None admin] RBAC: Authorizing `identity:get_project(project_id=e27e1edc913f4b99a1b0be6ab9d16b6f)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:20.891138 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49379ae-1eaf-4b96-99dd-8e0bd0cdc228 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:20.900774 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 193/388] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:34:20 2022] GET /identity/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:20.903268 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49379ae-1eaf-4b96-99dd-8e0bd0cdc228 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:20.931863 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e1a91-ac42-4877-bf7b-29c6407fbf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:20.932735 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e1a91-ac42-4877-bf7b-29c6407fbf24 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:20.933067 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e1a91-ac42-4877-bf7b-29c6407fbf24 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:20.933348 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e1a91-ac42-4877-bf7b-29c6407fbf24 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:20.959130 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e1a91-ac42-4877-bf7b-29c6407fbf24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:20.968950 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-414e1a91-ac42-4877-bf7b-29c6407fbf24 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 14:34:20.968950 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:20.968950 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:20.968950 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:20.968950 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:20.968950 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:20.968950 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:20.968950 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:20.968950 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:20.968950 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:20.968950 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:20.968950 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:20.968950 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:20.968950 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:20.968950 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:20.968950 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:20.970033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:20.970033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:20.970033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:20.970033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:20.970033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:20.970033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:20.970033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:20.970033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:20.970033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:20.970033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:20.970033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:20.970033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:20.970033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:20.970033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:20.970033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:20.970033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:20.970882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:20.970882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:20.970882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:20.970882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:20.970882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:20.970882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:20.970882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:20.970882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:20.970882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:20.970882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:20.970882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 14:34:20.970882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:20.971877 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 196/389] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jul 20 14:34:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:34:20.977709 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e1a91-ac42-4877-bf7b-29c6407fbf24 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:21.086986 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e1a91-ac42-4877-bf7b-29c6407fbf24 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:21.099448 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:21.123668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f140a-623a-4307-b157-e196dfb7f9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:21.124519 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f140a-623a-4307-b157-e196dfb7f9b8 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:21.124903 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f140a-623a-4307-b157-e196dfb7f9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:21.125283 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f140a-623a-4307-b157-e196dfb7f9b8 None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:21.126153 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f140a-623a-4307-b157-e196dfb7f9b8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:21.126575 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f140a-623a-4307-b157-e196dfb7f9b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:21.128135 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f140a-623a-4307-b157-e196dfb7f9b8 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:21.138047 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 197/390] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:34:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:21.143948 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f140a-623a-4307-b157-e196dfb7f9b8 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:21.149008 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:21.193089 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49379ae-1eaf-4b96-99dd-8e0bd0cdc228 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:21.286623 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446307e-c63c-4b26-a570-49ea3e33c567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:21.287626 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446307e-c63c-4b26-a570-49ea3e33c567 None admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:21.287962 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446307e-c63c-4b26-a570-49ea3e33c567 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:21.288250 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446307e-c63c-4b26-a570-49ea3e33c567 None admin] PATH_INFO: `/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f/groups/fc8ddab7364d4c7ba121fe83f01187c2/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:21.292655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1446307e-c63c-4b26-a570-49ea3e33c567 None admin] RBAC: Authorizing `identity:create_grant(project_id=e27e1edc913f4b99a1b0be6ab9d16b6f, group_id=fc8ddab7364d4c7ba121fe83f01187c2, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:21.294125 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1446307e-c63c-4b26-a570-49ea3e33c567 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:21.326499 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 194/391] 10.222.0.15 () {58 vars in 1442 bytes} [Wed Jul 20 14:34:20 2022] PUT /identity/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f/groups/fc8ddab7364d4c7ba121fe83f01187c2/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:34:21.365765 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f140a-623a-4307-b157-e196dfb7f9b8 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:21.383780 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db321d-6b2a-4044-b531-fcd98c0e918a None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:21.384421 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db321d-6b2a-4044-b531-fcd98c0e918a None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:21.384895 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db321d-6b2a-4044-b531-fcd98c0e918a None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:21.386474 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 195/392] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 14:34:21.395997 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4aba0-7f25-4a6e-8fb5-46332f04cd3a None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:21.396319 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4aba0-7f25-4a6e-8fb5-46332f04cd3a None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:21.396554 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4aba0-7f25-4a6e-8fb5-46332f04cd3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:21.408403 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3559a809-3b8c-432b-a613-c8626c10c28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:21.409301 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3559a809-3b8c-432b-a613-c8626c10c28f None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:21.409584 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3559a809-3b8c-432b-a613-c8626c10c28f None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:21.409861 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3559a809-3b8c-432b-a613-c8626c10c28f None admin] PATH_INFO: `/v3/groups/fc8ddab7364d4c7ba121fe83f01187c2` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:21.418490 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3559a809-3b8c-432b-a613-c8626c10c28f None admin] RBAC: Authorizing `identity:get_group(group_id=fc8ddab7364d4c7ba121fe83f01187c2)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:21.420043 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3559a809-3b8c-432b-a613-c8626c10c28f None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:21.422634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 198/393] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Jul 20 14:34:21 2022] GET /identity/v3/groups/fc8ddab7364d4c7ba121fe83f01187c2 => generated 226 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:21.429403 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3559a809-3b8c-432b-a613-c8626c10c28f None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:21.437263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-65f4aba0-7f25-4a6e-8fb5-46332f04cd3a None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:21.441794 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-65f4aba0-7f25-4a6e-8fb5-46332f04cd3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:21.544366 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 196/394] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:21.544550 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:21.573249 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4aba0-7f25-4a6e-8fb5-46332f04cd3a None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:21.619254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3559a809-3b8c-432b-a613-c8626c10c28f None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:21.655245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:21.676295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7c84c-928b-4495-9b10-9b0e7114caf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:21.677195 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7c84c-928b-4495-9b10-9b0e7114caf5 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:21.677473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7c84c-928b-4495-9b10-9b0e7114caf5 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:21.677745 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7c84c-928b-4495-9b10-9b0e7114caf5 None admin] PATH_INFO: `/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:21.683829 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e7c84c-928b-4495-9b10-9b0e7114caf5 None admin] RBAC: Authorizing `identity:get_project(project_id=e27e1edc913f4b99a1b0be6ab9d16b6f)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:21.685299 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e7c84c-928b-4495-9b10-9b0e7114caf5 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:21.687001 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4aba0-7f25-4a6e-8fb5-46332f04cd3a None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:21.687382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 199/395] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:34:21 2022] GET /identity/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:34:21.692709 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7c84c-928b-4495-9b10-9b0e7114caf5 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:21.723667 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d34f48-46d8-4493-b907-abbd46d76a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:21.725518 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d34f48-46d8-4493-b907-abbd46d76a5d None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:21.725518 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d34f48-46d8-4493-b907-abbd46d76a5d None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:21.725518 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d34f48-46d8-4493-b907-abbd46d76a5d None admin] PATH_INFO: `/v3/roles/member` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:21.737736 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d34f48-46d8-4493-b907-abbd46d76a5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:21.749137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-a3d34f48-46d8-4493-b907-abbd46d76a5d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:34:21.749137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:21.749137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:21.749137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:21.749137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:21.749137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:21.749137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:21.749137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:21.749137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:21.749137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:21.749137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:21.749137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:21.749137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:21.749137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:21.749137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:21.749137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:21.750155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:21.750155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:21.750155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:21.750155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:21.750155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:21.750155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:21.750155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:21.750155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:21.750155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:21.750155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:21.750155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:21.750155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:21.750155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:21.750155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:21.750155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:21.750155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:21.750955 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:21.750955 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:21.750955 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:21.750955 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:21.750955 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:21.750955 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:21.750955 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:21.750955 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:21.750955 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:21.750955 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:21.750955 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:34:21.750955 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:21.752002 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 197/396] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:34:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:34:21.764790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d34f48-46d8-4493-b907-abbd46d76a5d None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:21.845600 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7c84c-928b-4495-9b10-9b0e7114caf5 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:21.848684 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:21.888624 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99415d16-e528-4deb-be69-7e2c194caac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:21.889906 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99415d16-e528-4deb-be69-7e2c194caac3 None admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:21.890372 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99415d16-e528-4deb-be69-7e2c194caac3 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:21.890993 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99415d16-e528-4deb-be69-7e2c194caac3 None admin] PATH_INFO: `/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f/groups/fc8ddab7364d4c7ba121fe83f01187c2/roles/b74189174ce24c1382712858dc35f3ce` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:21.893905 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99415d16-e528-4deb-be69-7e2c194caac3 None admin] RBAC: Authorizing `identity:create_grant(project_id=e27e1edc913f4b99a1b0be6ab9d16b6f, group_id=fc8ddab7364d4c7ba121fe83f01187c2, role_id=b74189174ce24c1382712858dc35f3ce)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:21.895854 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99415d16-e528-4deb-be69-7e2c194caac3 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:21.922988 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 200/397] 10.222.0.15 () {58 vars in 1442 bytes} [Wed Jul 20 14:34:21 2022] PUT /identity/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f/groups/fc8ddab7364d4c7ba121fe83f01187c2/roles/b74189174ce24c1382712858dc35f3ce => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 14:34:21.952212 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b5d57-7ad0-4b81-ae56-6a85d47d92b0 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:21.952938 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b5d57-7ad0-4b81-ae56-6a85d47d92b0 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:21.953457 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b5d57-7ad0-4b81-ae56-6a85d47d92b0 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:21.955551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 201/398] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 14:34:21.967123 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae73082-b0bf-4a80-81c3-333ddb9f4e97 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:21.967425 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae73082-b0bf-4a80-81c3-333ddb9f4e97 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:21.967688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae73082-b0bf-4a80-81c3-333ddb9f4e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:21.993447 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-5ae73082-b0bf-4a80-81c3-333ddb9f4e97 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:21.994881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-5ae73082-b0bf-4a80-81c3-333ddb9f4e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:22.001627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d34f48-46d8-4493-b907-abbd46d76a5d None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:22.055862 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:22.068213 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f9bfbb-1c67-44a7-94db-503dbdb7ecdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:22.068915 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9bfbb-1c67-44a7-94db-503dbdb7ecdd None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:22.069201 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9bfbb-1c67-44a7-94db-503dbdb7ecdd None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:22.069476 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9bfbb-1c67-44a7-94db-503dbdb7ecdd None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:22.070322 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f9bfbb-1c67-44a7-94db-503dbdb7ecdd None admin] RBAC: Adding query filter params (name=member) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:22.070642 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f9bfbb-1c67-44a7-94db-503dbdb7ecdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:22.073587 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f9bfbb-1c67-44a7-94db-503dbdb7ecdd None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:22.089395 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 198/399] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jul 20 14:34:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:22.093640 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f9bfbb-1c67-44a7-94db-503dbdb7ecdd None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:22.122811 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 202/400] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 14:34:22.158567 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae73082-b0bf-4a80-81c3-333ddb9f4e97 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:22.234835 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f9bfbb-1c67-44a7-94db-503dbdb7ecdd None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:22.278034 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc39f0e-2765-4e68-8fe3-571ca83c2eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:22.279115 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:22.279855 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc39f0e-2765-4e68-8fe3-571ca83c2eb8 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:22.280206 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc39f0e-2765-4e68-8fe3-571ca83c2eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:22.280552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc39f0e-2765-4e68-8fe3-571ca83c2eb8 None admin] PATH_INFO: `/v3/groups/fc8ddab7364d4c7ba121fe83f01187c2` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:22.282032 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc39f0e-2765-4e68-8fe3-571ca83c2eb8 None admin] RBAC: Authorizing `identity:get_group(group_id=fc8ddab7364d4c7ba121fe83f01187c2)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:22.283614 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc39f0e-2765-4e68-8fe3-571ca83c2eb8 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:22.286096 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 199/401] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Jul 20 14:34:22 2022] GET /identity/v3/groups/fc8ddab7364d4c7ba121fe83f01187c2 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:22.292002 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc39f0e-2765-4e68-8fe3-571ca83c2eb8 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:22.298051 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae73082-b0bf-4a80-81c3-333ddb9f4e97 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:22.371243 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68a1f7-508b-4635-b32e-d8f3c0d86955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:22.372129 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68a1f7-508b-4635-b32e-d8f3c0d86955 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:22.372411 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68a1f7-508b-4635-b32e-d8f3c0d86955 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:22.372684 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68a1f7-508b-4635-b32e-d8f3c0d86955 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:22.394039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d68a1f7-508b-4635-b32e-d8f3c0d86955 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:22.403417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:22.413712 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-9d68a1f7-508b-4635-b32e-d8f3c0d86955 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 14:34:22.413712 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:22.413712 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:22.413712 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:22.413712 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:22.413712 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:22.413712 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:22.413712 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:22.413712 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:22.413712 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:22.413712 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:22.413712 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:22.413712 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:22.413712 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:22.413712 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:22.413712 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:22.414908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:22.414908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:22.414908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:22.414908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:22.414908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:22.414908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:22.414908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:22.414908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:22.414908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:22.414908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:22.414908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:22.414908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:22.414908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:22.414908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:22.414908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:22.414908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:22.415805 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:22.415805 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:22.415805 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:22.415805 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:22.415805 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:22.415805 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:22.415805 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:22.415805 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:22.415805 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:22.415805 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:22.415805 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 14:34:22.415805 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:22.418656 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 203/402] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jul 20 14:34:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:34:22.420899 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68a1f7-508b-4635-b32e-d8f3c0d86955 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:22.473990 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc39f0e-2765-4e68-8fe3-571ca83c2eb8 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:22.557287 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b8843-591d-487a-b3a9-45e15111ca1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:22.558584 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b8843-591d-487a-b3a9-45e15111ca1b None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:22.559090 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b8843-591d-487a-b3a9-45e15111ca1b None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:22.559587 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b8843-591d-487a-b3a9-45e15111ca1b None admin] PATH_INFO: `/v3/projects/888e6ef050b643db90e7280d61f3e3e8` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:22.563727 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2b8843-591d-487a-b3a9-45e15111ca1b None admin] RBAC: Authorizing `identity:get_project(project_id=888e6ef050b643db90e7280d61f3e3e8)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:22.565408 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2b8843-591d-487a-b3a9-45e15111ca1b None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:22.567027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68a1f7-508b-4635-b32e-d8f3c0d86955 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:22.567935 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 200/403] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:34:22 2022] GET /identity/v3/projects/888e6ef050b643db90e7280d61f3e3e8 => generated 302 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:34:22.573875 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b8843-591d-487a-b3a9-45e15111ca1b None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:22.625023 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ec6fd-348b-487a-8c34-b0fc912bc51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:22.625700 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ec6fd-348b-487a-8c34-b0fc912bc51d None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:22.625988 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ec6fd-348b-487a-8c34-b0fc912bc51d None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:22.626260 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ec6fd-348b-487a-8c34-b0fc912bc51d None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:22.627023 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ec6fd-348b-487a-8c34-b0fc912bc51d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:22.627350 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ec6fd-348b-487a-8c34-b0fc912bc51d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:22.629287 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ec6fd-348b-487a-8c34-b0fc912bc51d None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:22.641120 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 204/404] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:34:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:22.643114 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced49f8-db8a-4c19-a9b6-b5912fd5bcf5 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:22.643412 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced49f8-db8a-4c19-a9b6-b5912fd5bcf5 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:22.643693 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced49f8-db8a-4c19-a9b6-b5912fd5bcf5 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:22.645773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 205/405] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:22.647848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced49f8-db8a-4c19-a9b6-b5912fd5bcf5 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:22.670361 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:22.715934 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b8843-591d-487a-b3a9-45e15111ca1b None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:22.745758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:22.752952 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f74d0-ad0a-48b5-a1a2-aa2958fc9bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:22.754057 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f74d0-ad0a-48b5-a1a2-aa2958fc9bcd None admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:22.754519 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f74d0-ad0a-48b5-a1a2-aa2958fc9bcd None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:22.754931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f74d0-ad0a-48b5-a1a2-aa2958fc9bcd None admin] PATH_INFO: `/v3/projects/888e6ef050b643db90e7280d61f3e3e8/groups/fc8ddab7364d4c7ba121fe83f01187c2/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:22.757177 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f74d0-ad0a-48b5-a1a2-aa2958fc9bcd None admin] RBAC: Authorizing `identity:create_grant(project_id=888e6ef050b643db90e7280d61f3e3e8, group_id=fc8ddab7364d4c7ba121fe83f01187c2, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:22.758797 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f74d0-ad0a-48b5-a1a2-aa2958fc9bcd None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:22.763340 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced49f8-db8a-4c19-a9b6-b5912fd5bcf5 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:22.783278 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 201/406] 10.222.0.15 () {58 vars in 1442 bytes} [Wed Jul 20 14:34:22 2022] PUT /identity/v3/projects/888e6ef050b643db90e7280d61f3e3e8/groups/fc8ddab7364d4c7ba121fe83f01187c2/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 14:34:22.785255 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65d460-94b3-4d6e-8a8e-1333c9a87015 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:22.785849 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65d460-94b3-4d6e-8a8e-1333c9a87015 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:22.786293 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65d460-94b3-4d6e-8a8e-1333c9a87015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:22.807121 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c6a4e-646a-4640-8b6a-02743da0e9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:22.807121 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c6a4e-646a-4640-8b6a-02743da0e9a3 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:22.807121 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c6a4e-646a-4640-8b6a-02743da0e9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:22.807121 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c6a4e-646a-4640-8b6a-02743da0e9a3 None admin] PATH_INFO: `/v3/groups/fc8ddab7364d4c7ba121fe83f01187c2` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:22.807544 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c6a4e-646a-4640-8b6a-02743da0e9a3 None admin] RBAC: Authorizing `identity:get_group(group_id=fc8ddab7364d4c7ba121fe83f01187c2)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:22.807544 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c6a4e-646a-4640-8b6a-02743da0e9a3 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:22.809605 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 206/407] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Jul 20 14:34:22 2022] GET /identity/v3/groups/fc8ddab7364d4c7ba121fe83f01187c2 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:22.814953 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c6a4e-646a-4640-8b6a-02743da0e9a3 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:22.822852 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-ed65d460-94b3-4d6e-8a8e-1333c9a87015 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:22.824479 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-ed65d460-94b3-4d6e-8a8e-1333c9a87015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:22.909684 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:22.931076 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c6a4e-646a-4640-8b6a-02743da0e9a3 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:22.952946 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 202/408] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:22.966225 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3954c3-2225-4e09-b117-5037de0d5b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:22.967189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3954c3-2225-4e09-b117-5037de0d5b08 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:22.967481 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3954c3-2225-4e09-b117-5037de0d5b08 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:22.967756 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3954c3-2225-4e09-b117-5037de0d5b08 None admin] PATH_INFO: `/v3/projects/888e6ef050b643db90e7280d61f3e3e8` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:22.969024 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3954c3-2225-4e09-b117-5037de0d5b08 None admin] RBAC: Authorizing `identity:get_project(project_id=888e6ef050b643db90e7280d61f3e3e8)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:22.970527 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3954c3-2225-4e09-b117-5037de0d5b08 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:22.972644 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 207/409] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:34:22 2022] GET /identity/v3/projects/888e6ef050b643db90e7280d61f3e3e8 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:22.978510 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65d460-94b3-4d6e-8a8e-1333c9a87015 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:22.988505 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3954c3-2225-4e09-b117-5037de0d5b08 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:23.114544 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3954c3-2225-4e09-b117-5037de0d5b08 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:23.154742 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316bc6f1-731b-4662-b4e2-0023f76b7042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:23.156563 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bc6f1-731b-4662-b4e2-0023f76b7042 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:23.157037 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bc6f1-731b-4662-b4e2-0023f76b7042 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:23.157517 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bc6f1-731b-4662-b4e2-0023f76b7042 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:23.175376 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316bc6f1-731b-4662-b4e2-0023f76b7042 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:23.180663 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65d460-94b3-4d6e-8a8e-1333c9a87015 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:23.185688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-316bc6f1-731b-4662-b4e2-0023f76b7042 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:34:23.185688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:23.185688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:23.185688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:23.185688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:23.185688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:23.185688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:23.185688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:23.185688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:23.185688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:23.185688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:23.185688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:23.185688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:23.185688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:23.185688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:23.185688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:23.186680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:23.186680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:23.186680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:23.186680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:23.186680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:23.186680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:23.186680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:23.186680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:23.186680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:23.186680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:23.186680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:23.186680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:23.186680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:23.186680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:23.186680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:23.186680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:23.187939 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:23.187939 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:23.187939 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:23.187939 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:23.187939 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:23.187939 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:23.187939 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:23.187939 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:23.187939 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:23.187939 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:23.187939 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:34:23.187939 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:23.187939 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 208/410] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:34:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:23.193635 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316bc6f1-731b-4662-b4e2-0023f76b7042 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:23.210741 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:23.212830 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:23.231161 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d55c1c1-2ccc-46bf-842b-d4e26534c76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:23.232531 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55c1c1-2ccc-46bf-842b-d4e26534c76c None admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:23.233018 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55c1c1-2ccc-46bf-842b-d4e26534c76c None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:23.233528 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55c1c1-2ccc-46bf-842b-d4e26534c76c None admin] PATH_INFO: `/v3/projects/888e6ef050b643db90e7280d61f3e3e8/groups/fc8ddab7364d4c7ba121fe83f01187c2/roles/b74189174ce24c1382712858dc35f3ce` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:23.235963 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d55c1c1-2ccc-46bf-842b-d4e26534c76c None admin] RBAC: Authorizing `identity:create_grant(project_id=888e6ef050b643db90e7280d61f3e3e8, group_id=fc8ddab7364d4c7ba121fe83f01187c2, role_id=b74189174ce24c1382712858dc35f3ce)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:23.238244 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d55c1c1-2ccc-46bf-842b-d4e26534c76c None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:23.261893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 203/411] 10.222.0.15 () {58 vars in 1442 bytes} [Wed Jul 20 14:34:22 2022] PUT /identity/v3/projects/888e6ef050b643db90e7280d61f3e3e8/groups/fc8ddab7364d4c7ba121fe83f01187c2/roles/b74189174ce24c1382712858dc35f3ce => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:34:23.330342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316bc6f1-731b-4662-b4e2-0023f76b7042 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:23.371959 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cfdaee-0658-4706-b025-30fe6c44af68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:23.372679 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cfdaee-0658-4706-b025-30fe6c44af68 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:23.372962 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cfdaee-0658-4706-b025-30fe6c44af68 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:23.373240 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cfdaee-0658-4706-b025-30fe6c44af68 None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:23.374017 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cfdaee-0658-4706-b025-30fe6c44af68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:23.374331 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cfdaee-0658-4706-b025-30fe6c44af68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:23.375757 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cfdaee-0658-4706-b025-30fe6c44af68 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:23.385013 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 209/412] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:34:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:34:23.390971 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d55c1c1-2ccc-46bf-842b-d4e26534c76c None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:23.530518 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d55c1c1-2ccc-46bf-842b-d4e26534c76c None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:23.568121 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517d499-d9fe-49ba-8b99-19bddb7bd662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:23.569429 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517d499-d9fe-49ba-8b99-19bddb7bd662 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:23.569997 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517d499-d9fe-49ba-8b99-19bddb7bd662 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:23.570475 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517d499-d9fe-49ba-8b99-19bddb7bd662 None admin] PATH_INFO: `/v3/groups/50b15137c74c40df9d5f00a6a168e3cd` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:23.571976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a517d499-d9fe-49ba-8b99-19bddb7bd662 None admin] RBAC: Authorizing `identity:get_group(group_id=50b15137c74c40df9d5f00a6a168e3cd)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:23.573747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a517d499-d9fe-49ba-8b99-19bddb7bd662 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:23.576201 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 204/413] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Jul 20 14:34:23 2022] GET /identity/v3/groups/50b15137c74c40df9d5f00a6a168e3cd => generated 229 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:23.583080 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cfdaee-0658-4706-b025-30fe6c44af68 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:23.708170 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:23.715496 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cfdaee-0658-4706-b025-30fe6c44af68 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:23.753190 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ccf3c3-8f46-4ef6-94e3-9edb2d3bc279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:23.754006 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ccf3c3-8f46-4ef6-94e3-9edb2d3bc279 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:23.754290 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ccf3c3-8f46-4ef6-94e3-9edb2d3bc279 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:23.754575 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ccf3c3-8f46-4ef6-94e3-9edb2d3bc279 None admin] PATH_INFO: `/v3/projects/3ca83facd1c8420dacb9e91e8abe0fd4` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:23.755827 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ccf3c3-8f46-4ef6-94e3-9edb2d3bc279 None admin] RBAC: Authorizing `identity:get_project(project_id=3ca83facd1c8420dacb9e91e8abe0fd4)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:23.757373 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ccf3c3-8f46-4ef6-94e3-9edb2d3bc279 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:23.759564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 210/414] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:34:23 2022] GET /identity/v3/projects/3ca83facd1c8420dacb9e91e8abe0fd4 => generated 344 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:23.765388 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517d499-d9fe-49ba-8b99-19bddb7bd662 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:23.829945 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb99c29-9f02-41ca-b984-922f65291e5d None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:23.830285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb99c29-9f02-41ca-b984-922f65291e5d None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:23.830514 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb99c29-9f02-41ca-b984-922f65291e5d None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:23.832181 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 211/415] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:23.837657 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5a978-394c-46c3-bb83-cda9daf97bb7 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:23.837941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5a978-394c-46c3-bb83-cda9daf97bb7 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:23.838230 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5a978-394c-46c3-bb83-cda9daf97bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:23.839211 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:23.859775 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-50f5a978-394c-46c3-bb83-cda9daf97bb7 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:23.861368 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-50f5a978-394c-46c3-bb83-cda9daf97bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:23.915354 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517d499-d9fe-49ba-8b99-19bddb7bd662 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:23.934211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 212/416] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 14:34:23.958695 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5a978-394c-46c3-bb83-cda9daf97bb7 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:23.978932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f5392-9579-4595-bf98-9628624bef5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:23.980161 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f5392-9579-4595-bf98-9628624bef5e None admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:23.980332 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f5392-9579-4595-bf98-9628624bef5e None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:23.980514 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f5392-9579-4595-bf98-9628624bef5e None admin] PATH_INFO: `/v3/projects/3ca83facd1c8420dacb9e91e8abe0fd4/groups/50b15137c74c40df9d5f00a6a168e3cd/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:23.985384 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f5392-9579-4595-bf98-9628624bef5e None admin] RBAC: Authorizing `identity:create_grant(project_id=3ca83facd1c8420dacb9e91e8abe0fd4, group_id=50b15137c74c40df9d5f00a6a168e3cd, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:23.986863 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f5392-9579-4595-bf98-9628624bef5e None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:24.010747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 205/417] 10.222.0.15 () {58 vars in 1442 bytes} [Wed Jul 20 14:34:23 2022] PUT /identity/v3/projects/3ca83facd1c8420dacb9e91e8abe0fd4/groups/50b15137c74c40df9d5f00a6a168e3cd/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:34:24.052312 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:24.085716 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5a978-394c-46c3-bb83-cda9daf97bb7 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:24.123514 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9773f0-dd2a-43da-a5b4-1d2478dba7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:24.124352 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9773f0-dd2a-43da-a5b4-1d2478dba7af None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:24.124634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9773f0-dd2a-43da-a5b4-1d2478dba7af None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:24.124906 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9773f0-dd2a-43da-a5b4-1d2478dba7af None admin] PATH_INFO: `/v3/roles/member` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:24.137499 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9773f0-dd2a-43da-a5b4-1d2478dba7af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:24.145800 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-7e9773f0-dd2a-43da-a5b4-1d2478dba7af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:34:24.145800 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:24.145800 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:24.145800 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:24.145800 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:24.145800 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:24.145800 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:24.145800 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:24.145800 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:24.145800 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:24.145800 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:24.145800 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:24.145800 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:24.145800 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:24.145800 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:24.145800 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:24.146782 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:24.146782 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:24.146782 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:24.146782 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:24.146782 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:24.146782 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:24.146782 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:24.146782 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:24.146782 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:24.146782 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:24.146782 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:24.146782 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:24.146782 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:24.146782 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:24.146782 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:24.146782 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:24.147623 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:24.147623 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:24.147623 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:24.147623 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:24.147623 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:24.147623 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:24.147623 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:24.147623 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:24.147623 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:24.147623 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:24.147623 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:34:24.147623 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:24.147623 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 213/418] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:34:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:24.153625 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f5392-9579-4595-bf98-9628624bef5e None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:24.275275 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5f975-1cb9-4731-b580-f9b67c10d86d None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:24.275760 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5f975-1cb9-4731-b580-f9b67c10d86d None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:24.275760 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5f975-1cb9-4731-b580-f9b67c10d86d None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:24.278642 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 214/419] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 14:34:24.283961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82f594-f620-412a-a8c7-c5dda66fee7a None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:24.284243 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82f594-f620-412a-a8c7-c5dda66fee7a None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:24.284521 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82f594-f620-412a-a8c7-c5dda66fee7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:24.292837 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f5392-9579-4595-bf98-9628624bef5e None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:24.310100 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:24.317482 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-2f82f594-f620-412a-a8c7-c5dda66fee7a None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:24.318705 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-2f82f594-f620-412a-a8c7-c5dda66fee7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:24.352642 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3fb1c0-8cbd-4570-8bc6-06e5d4d3eee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:24.353347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fb1c0-8cbd-4570-8bc6-06e5d4d3eee4 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:24.353628 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fb1c0-8cbd-4570-8bc6-06e5d4d3eee4 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:24.353966 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fb1c0-8cbd-4570-8bc6-06e5d4d3eee4 None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:24.354743 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3fb1c0-8cbd-4570-8bc6-06e5d4d3eee4 None admin] RBAC: Adding query filter params (name=member) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:24.355305 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3fb1c0-8cbd-4570-8bc6-06e5d4d3eee4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:24.356728 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3fb1c0-8cbd-4570-8bc6-06e5d4d3eee4 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:24.375483 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 206/420] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jul 20 14:34:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:24.383548 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3fb1c0-8cbd-4570-8bc6-06e5d4d3eee4 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:24.444189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 215/421] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 14:34:24.467068 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f82f594-f620-412a-a8c7-c5dda66fee7a None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:24.529649 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:24.593833 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f82f594-f620-412a-a8c7-c5dda66fee7a None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:24.615691 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3fb1c0-8cbd-4570-8bc6-06e5d4d3eee4 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:24.645146 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe323f-ad71-4cb6-83f7-15e19e5af74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:24.646762 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe323f-ad71-4cb6-83f7-15e19e5af74a None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:24.647267 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe323f-ad71-4cb6-83f7-15e19e5af74a None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:24.647766 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe323f-ad71-4cb6-83f7-15e19e5af74a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:24.659605 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe323f-ad71-4cb6-83f7-15e19e5af74a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:24.670049 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-25fe323f-ad71-4cb6-83f7-15e19e5af74a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 14:34:24.670049 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:24.670049 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:24.670049 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:24.670049 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:24.670049 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:24.670049 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:24.670049 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:24.670049 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:24.670049 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:24.670049 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:24.670049 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:24.670049 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:24.670049 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:24.670049 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:24.670049 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:24.670961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:24.670961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:24.670961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:24.670961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:24.670961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:24.670961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:24.670961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:24.670961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:24.670961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:24.670961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:24.670961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:24.670961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:24.670961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:24.670961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:24.670961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:24.670961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:24.671929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:24.671929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:24.671929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:24.671929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:24.671929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:24.671929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:24.671929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:24.671929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:24.671929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:24.671929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:24.671929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 14:34:24.671929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:24.674353 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 216/422] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jul 20 14:34:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:34:24.680863 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe323f-ad71-4cb6-83f7-15e19e5af74a None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:24.688840 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e377352-782b-4c8e-941c-54d78098b332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:24.689620 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e377352-782b-4c8e-941c-54d78098b332 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:24.689911 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e377352-782b-4c8e-941c-54d78098b332 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:24.690271 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e377352-782b-4c8e-941c-54d78098b332 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:24.765996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:24.805248 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e377352-782b-4c8e-941c-54d78098b332 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:24.806909 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e377352-782b-4c8e-941c-54d78098b332 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:24.867357 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe323f-ad71-4cb6-83f7-15e19e5af74a None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:24.909846 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:24.912488 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 207/423] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:34:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:34:24.919022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e377352-782b-4c8e-941c-54d78098b332 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:24.927994 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca58cf-fb15-430b-8083-3a6b1e03c351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:24.929011 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca58cf-fb15-430b-8083-3a6b1e03c351 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:24.929519 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca58cf-fb15-430b-8083-3a6b1e03c351 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:24.930061 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca58cf-fb15-430b-8083-3a6b1e03c351 None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:24.931022 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ca58cf-fb15-430b-8083-3a6b1e03c351 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:24.931900 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ca58cf-fb15-430b-8083-3a6b1e03c351 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:24.933507 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ca58cf-fb15-430b-8083-3a6b1e03c351 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:24.944453 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 217/424] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:34:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:24.952935 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca58cf-fb15-430b-8083-3a6b1e03c351 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:25.038851 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:25.072735 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e377352-782b-4c8e-941c-54d78098b332 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:25.099489 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca58cf-fb15-430b-8083-3a6b1e03c351 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:25.128588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715bfcea-a15f-4c3f-8803-1bfeee63c075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:25.129469 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715bfcea-a15f-4c3f-8803-1bfeee63c075 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:25.129747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715bfcea-a15f-4c3f-8803-1bfeee63c075 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:25.130023 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715bfcea-a15f-4c3f-8803-1bfeee63c075 None admin] PATH_INFO: `/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:25.132925 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e92bfe-35d6-48ea-8653-9df5983ccf67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:25.133582 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e92bfe-35d6-48ea-8653-9df5983ccf67 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:25.133861 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e92bfe-35d6-48ea-8653-9df5983ccf67 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:25.134136 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e92bfe-35d6-48ea-8653-9df5983ccf67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:25.135790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715bfcea-a15f-4c3f-8803-1bfeee63c075 None admin] RBAC: Authorizing `identity:get_project(project_id=e27e1edc913f4b99a1b0be6ab9d16b6f)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:25.137550 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715bfcea-a15f-4c3f-8803-1bfeee63c075 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:25.139547 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 208/425] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:34:24 2022] GET /identity/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:34:25.145186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715bfcea-a15f-4c3f-8803-1bfeee63c075 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:25.191953 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:25.203523 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e92bfe-35d6-48ea-8653-9df5983ccf67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:25.211096 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e92bfe-35d6-48ea-8653-9df5983ccf67 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:25.279280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715bfcea-a15f-4c3f-8803-1bfeee63c075 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:25.308956 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 218/426] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:34:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:34:25.314590 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e92bfe-35d6-48ea-8653-9df5983ccf67 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:25.315441 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df277178-53ec-4590-a983-f29e861c6959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:25.316867 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df277178-53ec-4590-a983-f29e861c6959 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:25.317383 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df277178-53ec-4590-a983-f29e861c6959 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:25.317850 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df277178-53ec-4590-a983-f29e861c6959 None admin] PATH_INFO: `/v3/groups/fc8ddab7364d4c7ba121fe83f01187c2` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:25.319327 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df277178-53ec-4590-a983-f29e861c6959 None admin] RBAC: Authorizing `identity:get_group(group_id=fc8ddab7364d4c7ba121fe83f01187c2)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:25.321138 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df277178-53ec-4590-a983-f29e861c6959 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:25.327518 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 209/427] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Jul 20 14:34:25 2022] GET /identity/v3/groups/fc8ddab7364d4c7ba121fe83f01187c2 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:34:25.329367 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df277178-53ec-4590-a983-f29e861c6959 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:25.338619 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:25.417428 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:25.426656 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e92bfe-35d6-48ea-8653-9df5983ccf67 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:25.438963 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df277178-53ec-4590-a983-f29e861c6959 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:25.464352 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32294e35-ac3b-4b95-850d-08360ef5dd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:25.465695 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32294e35-ac3b-4b95-850d-08360ef5dd7b None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:25.466146 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32294e35-ac3b-4b95-850d-08360ef5dd7b None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:25.466586 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32294e35-ac3b-4b95-850d-08360ef5dd7b None admin] PATH_INFO: `/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:25.468265 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32294e35-ac3b-4b95-850d-08360ef5dd7b None admin] RBAC: Authorizing `identity:get_project(project_id=e27e1edc913f4b99a1b0be6ab9d16b6f)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:25.469892 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32294e35-ac3b-4b95-850d-08360ef5dd7b None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:25.472285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 219/428] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:34:25 2022] GET /identity/v3/projects/e27e1edc913f4b99a1b0be6ab9d16b6f => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:34:25.476065 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e404a4-55ac-4a53-a191-ea91a18b6844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:25.477363 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e404a4-55ac-4a53-a191-ea91a18b6844 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:25.477363 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e404a4-55ac-4a53-a191-ea91a18b6844 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:25.477363 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e404a4-55ac-4a53-a191-ea91a18b6844 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:25.477725 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e404a4-55ac-4a53-a191-ea91a18b6844 None admin] RBAC: Adding query filter params (group.id=fc8ddab7364d4c7ba121fe83f01187c2, role.id=7e394dd183b147ffa15b617f076bbb76, scope.project.id=e27e1edc913f4b99a1b0be6ab9d16b6f) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:25.478045 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e404a4-55ac-4a53-a191-ea91a18b6844 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:25.479102 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32294e35-ac3b-4b95-850d-08360ef5dd7b None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:25.479559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e404a4-55ac-4a53-a191-ea91a18b6844 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:25.486816 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 210/429] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Jul 20 14:34:25 2022] GET /identity/v3/role_assignments?group.id=fc8ddab7364d4c7ba121fe83f01187c2&scope.project.id=e27e1edc913f4b99a1b0be6ab9d16b6f&role.id=7e394dd183b147ffa15b617f076bbb76 => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:34:25.594003 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32294e35-ac3b-4b95-850d-08360ef5dd7b None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:25.620292 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e880716e-6f61-4fc5-ba85-deb0d0bd8f1e None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:25.620643 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e880716e-6f61-4fc5-ba85-deb0d0bd8f1e None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:25.620866 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e880716e-6f61-4fc5-ba85-deb0d0bd8f1e None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:25.622679 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 211/430] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:25.628286 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d586dad-2844-42fa-b8e6-cd1d6ec7caf8 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:25.628577 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d586dad-2844-42fa-b8e6-cd1d6ec7caf8 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:25.628861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d586dad-2844-42fa-b8e6-cd1d6ec7caf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:25.633491 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465c715-1ee0-4de8-9ec3-133bb9d8d512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:25.634722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465c715-1ee0-4de8-9ec3-133bb9d8d512 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:25.635253 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465c715-1ee0-4de8-9ec3-133bb9d8d512 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:25.635787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465c715-1ee0-4de8-9ec3-133bb9d8d512 None admin] PATH_INFO: `/v3/groups/fc8ddab7364d4c7ba121fe83f01187c2` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:25.637252 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6465c715-1ee0-4de8-9ec3-133bb9d8d512 None admin] RBAC: Authorizing `identity:get_group(group_id=fc8ddab7364d4c7ba121fe83f01187c2)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:25.639019 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6465c715-1ee0-4de8-9ec3-133bb9d8d512 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:25.641402 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 220/431] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Jul 20 14:34:25 2022] GET /identity/v3/groups/fc8ddab7364d4c7ba121fe83f01187c2 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:25.647418 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-7d586dad-2844-42fa-b8e6-cd1d6ec7caf8 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:25.647982 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465c715-1ee0-4de8-9ec3-133bb9d8d512 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:25.648822 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-7d586dad-2844-42fa-b8e6-cd1d6ec7caf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:25.671531 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:25.742449 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:25.747780 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 212/432] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 14:34:25.758793 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465c715-1ee0-4de8-9ec3-133bb9d8d512 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:25.784166 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d586dad-2844-42fa-b8e6-cd1d6ec7caf8 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:25.797587 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31061c0-aec2-4382-bd84-7e746e60b52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:25.798923 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31061c0-aec2-4382-bd84-7e746e60b52e None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:25.799369 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31061c0-aec2-4382-bd84-7e746e60b52e None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:25.799811 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31061c0-aec2-4382-bd84-7e746e60b52e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:25.800809 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31061c0-aec2-4382-bd84-7e746e60b52e None admin] RBAC: Adding query filter params (group.id=fc8ddab7364d4c7ba121fe83f01187c2, role.id=b74189174ce24c1382712858dc35f3ce, scope.project.id=e27e1edc913f4b99a1b0be6ab9d16b6f) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:25.801286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31061c0-aec2-4382-bd84-7e746e60b52e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:25.802907 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31061c0-aec2-4382-bd84-7e746e60b52e None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:25.811155 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 221/433] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Jul 20 14:34:25 2022] GET /identity/v3/role_assignments?group.id=fc8ddab7364d4c7ba121fe83f01187c2&scope.project.id=e27e1edc913f4b99a1b0be6ab9d16b6f&role.id=b74189174ce24c1382712858dc35f3ce => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:34:25.927327 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d586dad-2844-42fa-b8e6-cd1d6ec7caf8 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:25.971144 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144c8106-c015-4030-835d-f7a0450a74ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:25.971985 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c8106-c015-4030-835d-f7a0450a74ba None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:25.972272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c8106-c015-4030-835d-f7a0450a74ba None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:25.972549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c8106-c015-4030-835d-f7a0450a74ba None admin] PATH_INFO: `/v3/roles/member` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:25.985266 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144c8106-c015-4030-835d-f7a0450a74ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:25.993439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-144c8106-c015-4030-835d-f7a0450a74ba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:34:25.993439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:25.993439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:25.993439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:25.993439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:25.993439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:25.993439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:25.993439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:25.993439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:25.993439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:25.993439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:25.993439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:25.993439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:25.993439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:25.993439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:25.993439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:25.994377 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:25.994377 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:25.994377 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:25.994377 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:25.994377 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:25.994377 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:25.994377 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:25.994377 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:25.994377 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:25.994377 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:25.994377 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:25.994377 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:25.994377 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:25.994377 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:25.994377 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:25.994377 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:25.995232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:25.995232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:25.995232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:25.995232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:25.995232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:25.995232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:25.995232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:25.995232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:25.995232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:25.995232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:25.995232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 14:34:25.995232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:25.995825 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 213/434] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:34:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:26.001224 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31061c0-aec2-4382-bd84-7e746e60b52e None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:26.105196 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:26.124874 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31061c0-aec2-4382-bd84-7e746e60b52e None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:26.160241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43431feb-3bfe-4bb5-a2f3-4f06f98c0013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:26.160934 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43431feb-3bfe-4bb5-a2f3-4f06f98c0013 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:26.161265 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43431feb-3bfe-4bb5-a2f3-4f06f98c0013 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:26.161542 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43431feb-3bfe-4bb5-a2f3-4f06f98c0013 None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:26.162307 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43431feb-3bfe-4bb5-a2f3-4f06f98c0013 None admin] RBAC: Adding query filter params (name=member) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:26.162614 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43431feb-3bfe-4bb5-a2f3-4f06f98c0013 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:26.164089 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43431feb-3bfe-4bb5-a2f3-4f06f98c0013 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:26.174475 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 222/435] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jul 20 14:34:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:26.180885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144c8106-c015-4030-835d-f7a0450a74ba None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:26.220622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cd888-475f-4cec-ad78-abbb53583ff6 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:26.220622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cd888-475f-4cec-ad78-abbb53583ff6 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:26.220622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cd888-475f-4cec-ad78-abbb53583ff6 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:26.221260 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 223/436] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:26.226203 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5135379-010e-4594-9731-94a687bf9f36 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:26.226495 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5135379-010e-4594-9731-94a687bf9f36 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:26.226774 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5135379-010e-4594-9731-94a687bf9f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:26.248499 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-a5135379-010e-4594-9731-94a687bf9f36 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:26.250086 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-a5135379-010e-4594-9731-94a687bf9f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:26.262241 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:26.313112 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144c8106-c015-4030-835d-f7a0450a74ba None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:26.347199 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 224/437] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 14:34:26.355973 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ed9b6-f9e5-41b9-bf0a-dfb7f9f068ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:26.356713 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ed9b6-f9e5-41b9-bf0a-dfb7f9f068ea None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:26.357064 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ed9b6-f9e5-41b9-bf0a-dfb7f9f068ea None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:26.357403 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ed9b6-f9e5-41b9-bf0a-dfb7f9f068ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:26.364630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5135379-010e-4594-9731-94a687bf9f36 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:26.453494 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ed9b6-f9e5-41b9-bf0a-dfb7f9f068ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:26.454897 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ed9b6-f9e5-41b9-bf0a-dfb7f9f068ea None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:26.469626 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:26.502597 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5135379-010e-4594-9731-94a687bf9f36 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:26.519338 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:26.539016 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d24d66-51c7-48f4-8ff7-1a0674a664b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:26.539874 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d24d66-51c7-48f4-8ff7-1a0674a664b6 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:26.540149 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d24d66-51c7-48f4-8ff7-1a0674a664b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:26.540442 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d24d66-51c7-48f4-8ff7-1a0674a664b6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:26.551372 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d24d66-51c7-48f4-8ff7-1a0674a664b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:26.559635 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-84d24d66-51c7-48f4-8ff7-1a0674a664b6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 14:34:26.559635 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:26.559635 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:26.559635 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:26.559635 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:26.559635 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:26.559635 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:26.559635 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:26.559635 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:26.559635 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:26.559635 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:26.559635 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:26.559635 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:26.559635 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:26.559635 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:26.559635 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:26.569087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:26.569087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:26.569087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:26.569087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:26.569087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:26.569087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:26.569087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:26.569087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:26.569087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:26.569087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:26.569087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:26.569087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:26.569087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:26.569087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:26.569087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:26.569087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:26.570039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:26.570039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:26.570039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:26.570039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:26.570039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:26.570039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:26.570039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:26.570039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:26.570039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:26.570039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:26.570039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 14:34:26.570039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:26.570039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 225/438] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jul 20 14:34:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:34:26.570706 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 214/439] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:34:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:34:26.570776 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7506468-92f0-459e-bcb2-6469645e769b None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:26.570776 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7506468-92f0-459e-bcb2-6469645e769b None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:26.570776 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7506468-92f0-459e-bcb2-6469645e769b None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:26.570776 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 226/440] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:26.570776 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7506468-92f0-459e-bcb2-6469645e769b None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:26.571103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dadb30b-9c5f-42d9-b450-b51156e2dcb3 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:26.571103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dadb30b-9c5f-42d9-b450-b51156e2dcb3 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:26.571103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dadb30b-9c5f-42d9-b450-b51156e2dcb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:26.586939 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-6dadb30b-9c5f-42d9-b450-b51156e2dcb3 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:26.588170 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-6dadb30b-9c5f-42d9-b450-b51156e2dcb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:26.654906 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 215/441] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 14:34:26.656228 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dadb30b-9c5f-42d9-b450-b51156e2dcb3 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:26.681374 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7506468-92f0-459e-bcb2-6469645e769b None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:26.715784 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2526ac8-1811-49cd-8d08-4c541655dfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:26.716404 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2526ac8-1811-49cd-8d08-4c541655dfdb None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:26.716668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2526ac8-1811-49cd-8d08-4c541655dfdb None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:26.716939 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2526ac8-1811-49cd-8d08-4c541655dfdb None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:26.717696 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2526ac8-1811-49cd-8d08-4c541655dfdb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:26.718003 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2526ac8-1811-49cd-8d08-4c541655dfdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:26.719477 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2526ac8-1811-49cd-8d08-4c541655dfdb None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:26.721635 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:26.728753 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 227/442] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:34:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:34:26.729562 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2526ac8-1811-49cd-8d08-4c541655dfdb None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:26.748386 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:26.758037 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dadb30b-9c5f-42d9-b450-b51156e2dcb3 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:26.791366 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7adf4-f1a3-42d5-a96d-f8882a57fe7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:26.792234 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7adf4-f1a3-42d5-a96d-f8882a57fe7c None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:26.792513 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7adf4-f1a3-42d5-a96d-f8882a57fe7c None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:26.792792 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7adf4-f1a3-42d5-a96d-f8882a57fe7c None admin] PATH_INFO: `/v3/projects/888e6ef050b643db90e7280d61f3e3e8` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:26.794026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7adf4-f1a3-42d5-a96d-f8882a57fe7c None admin] RBAC: Authorizing `identity:get_project(project_id=888e6ef050b643db90e7280d61f3e3e8)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:26.795555 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7adf4-f1a3-42d5-a96d-f8882a57fe7c None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:26.797301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 216/443] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:34:26 2022] GET /identity/v3/projects/888e6ef050b643db90e7280d61f3e3e8 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:26.798197 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7adf4-f1a3-42d5-a96d-f8882a57fe7c None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:26.832056 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2526ac8-1811-49cd-8d08-4c541655dfdb None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:26.866361 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a6bf4-7895-4531-9389-bbfdbcdfd22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:26.867301 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a6bf4-7895-4531-9389-bbfdbcdfd22e None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:26.867595 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a6bf4-7895-4531-9389-bbfdbcdfd22e None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:26.867876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a6bf4-7895-4531-9389-bbfdbcdfd22e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:26.878184 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550a6bf4-7895-4531-9389-bbfdbcdfd22e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:26.885978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-550a6bf4-7895-4531-9389-bbfdbcdfd22e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:34:26.885978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:26.885978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:26.885978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:26.885978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:26.885978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:26.885978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:26.885978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:26.885978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:26.885978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:26.885978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:26.885978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:26.885978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:26.885978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:26.885978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:26.885978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:26.886876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:26.886876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:26.886876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:26.886876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:26.886876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:26.886876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:26.886876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:26.886876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:26.886876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:26.886876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:26.886876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:26.886876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:26.886876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:26.886876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:26.886876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:26.886876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:26.888730 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:26.888730 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:26.888730 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:26.888730 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:26.888730 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:26.888730 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:26.888730 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:26.888730 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:26.888730 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:26.888730 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:26.888730 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:34:26.888730 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:26.888730 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 228/444] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:34:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:26.888730 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a6bf4-7895-4531-9389-bbfdbcdfd22e None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:26.900245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7adf4-f1a3-42d5-a96d-f8882a57fe7c None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:26.935420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a91d9d-ca08-4d02-be24-cdbb02a0a48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:26.936318 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a91d9d-ca08-4d02-be24-cdbb02a0a48c None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:26.936604 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a91d9d-ca08-4d02-be24-cdbb02a0a48c None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:26.936880 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a91d9d-ca08-4d02-be24-cdbb02a0a48c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:26.947751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:26.964021 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:26.990168 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a6bf4-7895-4531-9389-bbfdbcdfd22e None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:27.007727 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a91d9d-ca08-4d02-be24-cdbb02a0a48c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:27.009219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a91d9d-ca08-4d02-be24-cdbb02a0a48c None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:27.026366 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa5035-9e6b-4591-991e-34179f74eaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:27.027271 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa5035-9e6b-4591-991e-34179f74eaa3 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:27.027566 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa5035-9e6b-4591-991e-34179f74eaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:27.027848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa5035-9e6b-4591-991e-34179f74eaa3 None admin] PATH_INFO: `/v3/groups/fc8ddab7364d4c7ba121fe83f01187c2` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:27.029066 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caa5035-9e6b-4591-991e-34179f74eaa3 None admin] RBAC: Authorizing `identity:get_group(group_id=fc8ddab7364d4c7ba121fe83f01187c2)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:27.030522 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caa5035-9e6b-4591-991e-34179f74eaa3 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:27.032206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 229/445] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Jul 20 14:34:26 2022] GET /identity/v3/groups/fc8ddab7364d4c7ba121fe83f01187c2 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:27.037256 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa5035-9e6b-4591-991e-34179f74eaa3 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:27.106513 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 217/446] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:34:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:34:27.107479 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a91d9d-ca08-4d02-be24-cdbb02a0a48c None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:27.137058 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa5035-9e6b-4591-991e-34179f74eaa3 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:27.162656 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:27.164236 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:27.174102 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febd3a6b-72aa-41f4-a800-64cb29c81956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:27.174791 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd3a6b-72aa-41f4-a800-64cb29c81956 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:27.175106 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd3a6b-72aa-41f4-a800-64cb29c81956 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:27.175386 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd3a6b-72aa-41f4-a800-64cb29c81956 None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:27.176159 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febd3a6b-72aa-41f4-a800-64cb29c81956 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:27.176471 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febd3a6b-72aa-41f4-a800-64cb29c81956 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:27.177870 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febd3a6b-72aa-41f4-a800-64cb29c81956 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:27.186974 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 230/447] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:34:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:27.187814 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febd3a6b-72aa-41f4-a800-64cb29c81956 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:27.209803 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a91d9d-ca08-4d02-be24-cdbb02a0a48c None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:27.247205 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e20ce-44a5-4d1e-8e77-9962e12f179d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:27.247791 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e20ce-44a5-4d1e-8e77-9962e12f179d None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:27.248071 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e20ce-44a5-4d1e-8e77-9962e12f179d None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:27.248350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e20ce-44a5-4d1e-8e77-9962e12f179d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:27.249179 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e20ce-44a5-4d1e-8e77-9962e12f179d None admin] RBAC: Adding query filter params (group.id=fc8ddab7364d4c7ba121fe83f01187c2, role.id=7e394dd183b147ffa15b617f076bbb76, scope.project.id=888e6ef050b643db90e7280d61f3e3e8) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:27.249488 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e20ce-44a5-4d1e-8e77-9962e12f179d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:27.250930 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e20ce-44a5-4d1e-8e77-9962e12f179d None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:27.257050 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 218/448] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Jul 20 14:34:27 2022] GET /identity/v3/role_assignments?group.id=fc8ddab7364d4c7ba121fe83f01187c2&scope.project.id=888e6ef050b643db90e7280d61f3e3e8&role.id=7e394dd183b147ffa15b617f076bbb76 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:27.257811 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e20ce-44a5-4d1e-8e77-9962e12f179d None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:27.303484 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febd3a6b-72aa-41f4-a800-64cb29c81956 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:27.343418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe36432-b842-4b9c-bc15-4fae7c161431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:27.344322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe36432-b842-4b9c-bc15-4fae7c161431 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:27.344600 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe36432-b842-4b9c-bc15-4fae7c161431 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:27.344872 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe36432-b842-4b9c-bc15-4fae7c161431 None admin] PATH_INFO: `/v3/projects/888e6ef050b643db90e7280d61f3e3e8` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:27.346175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe36432-b842-4b9c-bc15-4fae7c161431 None admin] RBAC: Authorizing `identity:get_project(project_id=888e6ef050b643db90e7280d61f3e3e8)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:27.347698 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe36432-b842-4b9c-bc15-4fae7c161431 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:27.349648 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 231/449] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:34:27 2022] GET /identity/v3/projects/888e6ef050b643db90e7280d61f3e3e8 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:27.355252 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe36432-b842-4b9c-bc15-4fae7c161431 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:27.372228 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e20ce-44a5-4d1e-8e77-9962e12f179d None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:27.401411 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:27.416309 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d136d1-732c-4093-8f7c-7f8c68141be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:27.416739 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d136d1-732c-4093-8f7c-7f8c68141be0 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:27.417022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d136d1-732c-4093-8f7c-7f8c68141be0 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:27.417302 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d136d1-732c-4093-8f7c-7f8c68141be0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:27.420983 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:27.474184 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe36432-b842-4b9c-bc15-4fae7c161431 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:27.501838 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d136d1-732c-4093-8f7c-7f8c68141be0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:27.503339 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d136d1-732c-4093-8f7c-7f8c68141be0 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:27.515911 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebb49e-d2fa-44ac-919a-880bff7653c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:27.516785 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebb49e-d2fa-44ac-919a-880bff7653c5 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:27.517062 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebb49e-d2fa-44ac-919a-880bff7653c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:27.517414 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebb49e-d2fa-44ac-919a-880bff7653c5 None admin] PATH_INFO: `/v3/groups/fc8ddab7364d4c7ba121fe83f01187c2` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:27.518517 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acebb49e-d2fa-44ac-919a-880bff7653c5 None admin] RBAC: Authorizing `identity:get_group(group_id=fc8ddab7364d4c7ba121fe83f01187c2)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:27.520017 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acebb49e-d2fa-44ac-919a-880bff7653c5 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:27.521818 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 232/450] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Jul 20 14:34:27 2022] GET /identity/v3/groups/fc8ddab7364d4c7ba121fe83f01187c2 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:34:27.527577 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebb49e-d2fa-44ac-919a-880bff7653c5 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:27.615072 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 219/451] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:34:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:34:27.621948 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d136d1-732c-4093-8f7c-7f8c68141be0 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:27.642312 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebb49e-d2fa-44ac-919a-880bff7653c5 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:27.646536 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:27.666044 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:27.677352 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c7e61-795f-4698-91b9-ffa9609d60c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:27.678021 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c7e61-795f-4698-91b9-ffa9609d60c6 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:27.678326 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c7e61-795f-4698-91b9-ffa9609d60c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:27.678582 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c7e61-795f-4698-91b9-ffa9609d60c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:27.679490 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819c7e61-795f-4698-91b9-ffa9609d60c6 None admin] RBAC: Adding query filter params (group.id=fc8ddab7364d4c7ba121fe83f01187c2, role.id=b74189174ce24c1382712858dc35f3ce, scope.project.id=888e6ef050b643db90e7280d61f3e3e8) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:27.679808 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819c7e61-795f-4698-91b9-ffa9609d60c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:27.681240 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819c7e61-795f-4698-91b9-ffa9609d60c6 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:27.686882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 233/452] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Jul 20 14:34:27 2022] GET /identity/v3/role_assignments?group.id=fc8ddab7364d4c7ba121fe83f01187c2&scope.project.id=888e6ef050b643db90e7280d61f3e3e8&role.id=b74189174ce24c1382712858dc35f3ce => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:27.726590 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d136d1-732c-4093-8f7c-7f8c68141be0 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:27.765667 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b8c7d-26b8-4b14-87d0-20f5d5ecafc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:27.766474 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b8c7d-26b8-4b14-87d0-20f5d5ecafc0 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:27.766752 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b8c7d-26b8-4b14-87d0-20f5d5ecafc0 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:27.767031 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b8c7d-26b8-4b14-87d0-20f5d5ecafc0 None admin] PATH_INFO: `/v3/projects/3ca83facd1c8420dacb9e91e8abe0fd4` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:27.768368 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b8c7d-26b8-4b14-87d0-20f5d5ecafc0 None admin] RBAC: Authorizing `identity:get_project(project_id=3ca83facd1c8420dacb9e91e8abe0fd4)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:27.769876 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b8c7d-26b8-4b14-87d0-20f5d5ecafc0 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:27.771706 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 220/453] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:34:27 2022] GET /identity/v3/projects/3ca83facd1c8420dacb9e91e8abe0fd4 => generated 344 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:27.777839 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c7e61-795f-4698-91b9-ffa9609d60c6 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:27.888941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c7e61-795f-4698-91b9-ffa9609d60c6 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:27.924114 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55778595-8278-4e1e-b491-0bd5b480c773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:27.924999 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55778595-8278-4e1e-b491-0bd5b480c773 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:27.925293 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55778595-8278-4e1e-b491-0bd5b480c773 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:27.925589 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55778595-8278-4e1e-b491-0bd5b480c773 None admin] PATH_INFO: `/v3/groups/50b15137c74c40df9d5f00a6a168e3cd` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:27.927001 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55778595-8278-4e1e-b491-0bd5b480c773 None admin] RBAC: Authorizing `identity:get_group(group_id=50b15137c74c40df9d5f00a6a168e3cd)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:27.928494 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55778595-8278-4e1e-b491-0bd5b480c773 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:27.930481 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 234/454] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Jul 20 14:34:27 2022] GET /identity/v3/groups/50b15137c74c40df9d5f00a6a168e3cd => generated 229 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:27.936418 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b8c7d-26b8-4b14-87d0-20f5d5ecafc0 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:28.028959 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:28.039639 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b8c7d-26b8-4b14-87d0-20f5d5ecafc0 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:28.073165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff948e3-54de-46e5-9f3f-659bab42280e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:28.073829 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff948e3-54de-46e5-9f3f-659bab42280e None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:28.074113 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff948e3-54de-46e5-9f3f-659bab42280e None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:28.074392 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff948e3-54de-46e5-9f3f-659bab42280e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:28.075282 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff948e3-54de-46e5-9f3f-659bab42280e None admin] RBAC: Adding query filter params (group.id=50b15137c74c40df9d5f00a6a168e3cd, role.id=49bd5abf255048a0abaa9538e309f516, scope.project.id=3ca83facd1c8420dacb9e91e8abe0fd4) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:28.075601 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff948e3-54de-46e5-9f3f-659bab42280e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:28.077046 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff948e3-54de-46e5-9f3f-659bab42280e None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:28.082696 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 221/455] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Jul 20 14:34:27 2022] GET /identity/v3/role_assignments?group.id=50b15137c74c40df9d5f00a6a168e3cd&scope.project.id=3ca83facd1c8420dacb9e91e8abe0fd4&role.id=49bd5abf255048a0abaa9538e309f516 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:30.638394 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c16ea-9a24-4109-bf41-4c52db282745 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:30.638953 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c16ea-9a24-4109-bf41-4c52db282745 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:30.638953 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c16ea-9a24-4109-bf41-4c52db282745 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:30.640595 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 235/456] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:30.646022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415e011-a2e4-48d0-b97e-8fc9e9e70534 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:30.646408 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415e011-a2e4-48d0-b97e-8fc9e9e70534 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:30.646588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415e011-a2e4-48d0-b97e-8fc9e9e70534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:30.672046 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-2415e011-a2e4-48d0-b97e-8fc9e9e70534 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:30.673635 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-2415e011-a2e4-48d0-b97e-8fc9e9e70534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:30.757756 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 222/457] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 14:34:30.806992 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c16ea-9a24-4109-bf41-4c52db282745 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:30.894590 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:30.937518 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011c119-2e9f-478d-9ab4-79727842d1ed None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:30.937833 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011c119-2e9f-478d-9ab4-79727842d1ed None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:30.938080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011c119-2e9f-478d-9ab4-79727842d1ed None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:30.939505 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 223/458] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 14:34:30.945298 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012f972-f719-4c52-a477-e1c8fed12b46 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:30.945586 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012f972-f719-4c52-a477-e1c8fed12b46 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:30.945868 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012f972-f719-4c52-a477-e1c8fed12b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:30.959820 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c16ea-9a24-4109-bf41-4c52db282745 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:30.963530 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-e012f972-f719-4c52-a477-e1c8fed12b46 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:30.964951 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-e012f972-f719-4c52-a477-e1c8fed12b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:31.002605 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff0690-f457-40d2-b382-384b4a342b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:31.003454 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff0690-f457-40d2-b382-384b4a342b53 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:31.003744 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff0690-f457-40d2-b382-384b4a342b53 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:31.004025 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff0690-f457-40d2-b382-384b4a342b53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:31.019120 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff0690-f457-40d2-b382-384b4a342b53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:31.020798 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff0690-f457-40d2-b382-384b4a342b53 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:31.039835 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 224/459] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:31.041587 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab7b7f-388f-48d1-8f49-98492ad15fd7 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:31.042096 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab7b7f-388f-48d1-8f49-98492ad15fd7 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:31.042569 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab7b7f-388f-48d1-8f49-98492ad15fd7 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:31.042696 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-f8ff0690-f457-40d2-b382-384b4a342b53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:31.042696 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:31.042696 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:31.042696 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:31.042696 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:31.042696 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:31.042696 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:31.042696 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:31.042696 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:31.042696 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:31.042696 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:31.042696 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:31.042696 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:31.042696 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:31.042696 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:31.043516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:31.043516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:31.043516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:31.043516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:31.043516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:31.043516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:31.043516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:31.043516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:31.043516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:31.043516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:31.043516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:31.043516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:31.043516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:31.043516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:31.043516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:31.043516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:31.043516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:31.044727 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:31.044727 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:31.044727 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:31.044727 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:31.044727 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 236/460] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:31.046707 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 225/461] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:31.051188 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff0690-f457-40d2-b382-384b4a342b53 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:31.052875 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b8ce3-8692-44bf-95c0-99893f59e6a5 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:31.053367 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b8ce3-8692-44bf-95c0-99893f59e6a5 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:31.053874 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b8ce3-8692-44bf-95c0-99893f59e6a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:31.070890 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-958b8ce3-8692-44bf-95c0-99893f59e6a5 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:31.072501 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-958b8ce3-8692-44bf-95c0-99893f59e6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:31.083361 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:31.151291 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 226/462] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 14:34:31.152538 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b8ce3-8692-44bf-95c0-99893f59e6a5 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:31.176849 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff0690-f457-40d2-b382-384b4a342b53 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:31.187508 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:31.222506 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115de1d-8315-4862-a1cf-3936f7d7b097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:31.223224 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115de1d-8315-4862-a1cf-3936f7d7b097 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:31.223515 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115de1d-8315-4862-a1cf-3936f7d7b097 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:31.223800 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115de1d-8315-4862-a1cf-3936f7d7b097 None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:31.224542 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115de1d-8315-4862-a1cf-3936f7d7b097 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:31.224857 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115de1d-8315-4862-a1cf-3936f7d7b097 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:31.226953 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115de1d-8315-4862-a1cf-3936f7d7b097 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:31.244846 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 237/463] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:31.245721 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115de1d-8315-4862-a1cf-3936f7d7b097 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:31.277539 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b8ce3-8692-44bf-95c0-99893f59e6a5 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:31.321152 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5e442-a4a7-4745-8411-9d6f1f75e748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:31.322301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5e442-a4a7-4745-8411-9d6f1f75e748 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:31.322589 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5e442-a4a7-4745-8411-9d6f1f75e748 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:31.322867 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5e442-a4a7-4745-8411-9d6f1f75e748 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:31.336785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc5e442-a4a7-4745-8411-9d6f1f75e748 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:31.338287 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc5e442-a4a7-4745-8411-9d6f1f75e748 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:31.341798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:31.363417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-dcc5e442-a4a7-4745-8411-9d6f1f75e748 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:31.363417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:31.363417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:31.363417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:31.363417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:31.363417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:31.363417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:31.363417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:31.363417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:31.363417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:31.363417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:31.363417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:31.363417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:31.363417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:31.363417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:31.364350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:31.364350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:31.364350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:31.364350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:31.364350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:31.364350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:31.364350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:31.364350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:31.364350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:31.364350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:31.364350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:31.364350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:31.364350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:31.364350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:31.364350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:31.364350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:31.364350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:31.365844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:31.365844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:31.365844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:31.365844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:31.365844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 227/464] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:31.365844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5e442-a4a7-4745-8411-9d6f1f75e748 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:31.367752 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115de1d-8315-4862-a1cf-3936f7d7b097 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:31.404035 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d67a6-37b5-4fbb-b20d-0aad3410b453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:31.405251 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d67a6-37b5-4fbb-b20d-0aad3410b453 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:31.405764 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d67a6-37b5-4fbb-b20d-0aad3410b453 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:31.406255 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d67a6-37b5-4fbb-b20d-0aad3410b453 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:31.419534 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001d67a6-37b5-4fbb-b20d-0aad3410b453 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:31.421340 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001d67a6-37b5-4fbb-b20d-0aad3410b453 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:31.441900 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-001d67a6-37b5-4fbb-b20d-0aad3410b453 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:31.441900 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:31.441900 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:31.441900 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:31.441900 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:31.441900 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:31.441900 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:31.441900 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:31.441900 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:31.441900 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:31.441900 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:31.441900 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:31.441900 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:31.441900 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:31.441900 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:31.442810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:31.442810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:31.442810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:31.442810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:31.442810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:31.442810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:31.442810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:31.442810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:31.442810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:31.442810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:31.442810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:31.442810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:31.442810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:31.442810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:31.442810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:31.442810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:31.442810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:31.443690 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:31.443690 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:31.443690 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:31.443690 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:31.450895 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 238/465] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:31.452154 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d67a6-37b5-4fbb-b20d-0aad3410b453 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:31.477158 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:31.488380 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5e442-a4a7-4745-8411-9d6f1f75e748 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:31.524334 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a71356-98b8-4395-b844-e9065171d7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:31.525017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a71356-98b8-4395-b844-e9065171d7e3 None admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:31.525309 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a71356-98b8-4395-b844-e9065171d7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:31.525588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a71356-98b8-4395-b844-e9065171d7e3 None admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:31.526424 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a71356-98b8-4395-b844-e9065171d7e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:31.527966 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a71356-98b8-4395-b844-e9065171d7e3 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:31.559076 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 228/466] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jul 20 14:34:31 2022] POST /identity/v3/users => generated 246 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:34:31.559076 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a71356-98b8-4395-b844-e9065171d7e3 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:31.575322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d67a6-37b5-4fbb-b20d-0aad3410b453 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:31.619384 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:31.626631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b73ba3-40b5-4bfb-ad0f-32811a1dd7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:31.627691 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b73ba3-40b5-4bfb-ad0f-32811a1dd7b7 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:31.627985 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b73ba3-40b5-4bfb-ad0f-32811a1dd7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:31.628268 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b73ba3-40b5-4bfb-ad0f-32811a1dd7b7 None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:31.629237 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b73ba3-40b5-4bfb-ad0f-32811a1dd7b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:31.629953 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b73ba3-40b5-4bfb-ad0f-32811a1dd7b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:31.631768 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b73ba3-40b5-4bfb-ad0f-32811a1dd7b7 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:31.647066 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 239/467] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:31.648744 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c05a58-45f5-446b-840e-510e7df583aa None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:31.649125 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c05a58-45f5-446b-840e-510e7df583aa None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:31.649565 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c05a58-45f5-446b-840e-510e7df583aa None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:31.651311 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 240/468] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:31.657898 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c05a58-45f5-446b-840e-510e7df583aa None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:31.686046 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a71356-98b8-4395-b844-e9065171d7e3 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:31.741270 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d8697e-d18e-4dd6-87e0-04ad23ed2e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:31.742169 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8697e-d18e-4dd6-87e0-04ad23ed2e93 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:31.743443 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8697e-d18e-4dd6-87e0-04ad23ed2e93 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:31.743443 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8697e-d18e-4dd6-87e0-04ad23ed2e93 None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:31.743623 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d8697e-d18e-4dd6-87e0-04ad23ed2e93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:31.744409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d8697e-d18e-4dd6-87e0-04ad23ed2e93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:31.745335 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d8697e-d18e-4dd6-87e0-04ad23ed2e93 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:31.763886 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:31.766156 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 229/469] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:31.767662 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7d6cb-da6d-4988-9959-ec22e2f03615 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:31.767958 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7d6cb-da6d-4988-9959-ec22e2f03615 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:31.768288 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7d6cb-da6d-4988-9959-ec22e2f03615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:31.791110 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-6ad7d6cb-da6d-4988-9959-ec22e2f03615 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:31.791110 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-6ad7d6cb-da6d-4988-9959-ec22e2f03615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:31.791338 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c05a58-45f5-446b-840e-510e7df583aa None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:31.828258 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625487ac-0f92-4a19-b357-9d55fe2c263e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:31.828942 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625487ac-0f92-4a19-b357-9d55fe2c263e None admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:31.829222 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625487ac-0f92-4a19-b357-9d55fe2c263e None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:31.829497 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625487ac-0f92-4a19-b357-9d55fe2c263e None admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:31.830218 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625487ac-0f92-4a19-b357-9d55fe2c263e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:31.831672 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625487ac-0f92-4a19-b357-9d55fe2c263e None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:31.855618 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 241/470] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jul 20 14:34:31 2022] POST /identity/v3/users => generated 248 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:34:31.856626 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625487ac-0f92-4a19-b357-9d55fe2c263e None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:31.875549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 230/471] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:31.877554 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c54e0-6b2a-4704-8029-73f8b63d5560 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:31.878057 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c54e0-6b2a-4704-8029-73f8b63d5560 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:31.878527 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c54e0-6b2a-4704-8029-73f8b63d5560 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:31.880538 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 231/472] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:31.887484 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddfae29-9a69-40a1-8362-2edba89974a5 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:31.888114 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddfae29-9a69-40a1-8362-2edba89974a5 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:31.888862 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddfae29-9a69-40a1-8362-2edba89974a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:31.910928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-bddfae29-9a69-40a1-8362-2edba89974a5 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:31.912590 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-bddfae29-9a69-40a1-8362-2edba89974a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:31.964165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:32.004075 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625487ac-0f92-4a19-b357-9d55fe2c263e None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:32.013676 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 232/473] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:32.014605 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddfae29-9a69-40a1-8362-2edba89974a5 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:32.062984 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c3066-d7f1-48da-bc5e-1781e3abbaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:32.064125 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c3066-d7f1-48da-bc5e-1781e3abbaf4 None admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:32.064408 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c3066-d7f1-48da-bc5e-1781e3abbaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:32.064687 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c3066-d7f1-48da-bc5e-1781e3abbaf4 None admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:32.065415 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c3066-d7f1-48da-bc5e-1781e3abbaf4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:32.066870 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c3066-d7f1-48da-bc5e-1781e3abbaf4 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:32.074949 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:32.090509 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 242/474] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jul 20 14:34:31 2022] POST /identity/v3/users => generated 248 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:34:32.092368 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c3066-d7f1-48da-bc5e-1781e3abbaf4 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:32.163777 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddfae29-9a69-40a1-8362-2edba89974a5 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:32.220321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d7927-ba53-491c-aa9c-255197db0705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:32.221972 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d7927-ba53-491c-aa9c-255197db0705 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:32.222264 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d7927-ba53-491c-aa9c-255197db0705 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:32.222546 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d7927-ba53-491c-aa9c-255197db0705 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:32.238476 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d7927-ba53-491c-aa9c-255197db0705 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:32.240719 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d7927-ba53-491c-aa9c-255197db0705 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:32.245924 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c3066-d7f1-48da-bc5e-1781e3abbaf4 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:32.266096 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-b34d7927-ba53-491c-aa9c-255197db0705 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:32.266096 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:32.266096 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:32.266096 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:32.266096 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:32.266096 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:32.266096 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:32.266096 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:32.266096 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:32.266096 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:32.266096 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:32.266096 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:32.266096 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:32.266096 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:32.266096 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:32.267017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:32.267017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:32.267017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:32.267017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:32.267017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:32.267017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:32.267017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:32.267017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:32.267017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:32.267017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:32.267017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:32.267017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:32.267017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:32.267017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:32.267017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:32.267017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:32.267017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:32.267885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:32.267885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:32.267885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:32.267885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:32.268673 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 233/475] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:32.274593 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d7927-ba53-491c-aa9c-255197db0705 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:32.292845 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152e743-4d94-44d2-9a94-49c4978558e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:32.293791 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152e743-4d94-44d2-9a94-49c4978558e7 None admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:32.294077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152e743-4d94-44d2-9a94-49c4978558e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:32.294358 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152e743-4d94-44d2-9a94-49c4978558e7 None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:32.295671 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152e743-4d94-44d2-9a94-49c4978558e7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:32.297589 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152e743-4d94-44d2-9a94-49c4978558e7 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:32.306621 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:32.307394 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.sql.core [None req-5152e743-4d94-44d2-9a94-49c4978558e7 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 14:34:32.307394 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 20 14:34:32.307394 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [parameters: {'id': '1b7d8d75672e41ccaa8d3b8f50df4450', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 20 14:34:32.307394 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86860) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 14:34:32.330107 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-5152e743-4d94-44d2-9a94-49c4978558e7 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 20 14:34:32.330107 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:32.330107 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 14:34:32.330107 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 14:34:32.330107 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 14:34:32.330107 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 14:34:32.330107 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 14:34:32.330107 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 14:34:32.330107 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 14:34:32.330107 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application conn.query(q) Jul 20 14:34:32.330107 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 14:34:32.330107 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 14:34:32.330107 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 14:34:32.330107 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application result.read() Jul 20 14:34:32.330107 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 14:34:32.331946 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 14:34:32.331946 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 14:34:32.331946 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 14:34:32.331946 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 14:34:32.331946 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 14:34:32.331946 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 14:34:32.331946 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 14:34:32.331946 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 14:34:32.331946 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:32.331946 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 14:34:32.331946 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:32.331946 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:32.331946 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 14:34:32.331946 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 14:34:32.331946 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 20 14:34:32.331946 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return ref.to_dict() Jul 20 14:34:32.331946 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 14:34:32.331946 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application next(self.gen) Jul 20 14:34:32.333589 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 14:34:32.333589 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application yield resource Jul 20 14:34:32.333589 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 14:34:32.333589 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application next(self.gen) Jul 20 14:34:32.333589 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 14:34:32.333589 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 14:34:32.333589 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 14:34:32.333589 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 14:34:32.333589 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 14:34:32.333589 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise self.value Jul 20 14:34:32.333589 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 14:34:32.333589 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 14:34:32.333589 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 14:34:32.333589 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application session.commit() Jul 20 14:34:32.333589 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 14:34:32.333589 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 14:34:32.333589 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 14:34:32.334560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 14:34:32.334560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 14:34:32.334560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application self.session.flush() Jul 20 14:34:32.334560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 14:34:32.334560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 14:34:32.334560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 14:34:32.334560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 14:34:32.334560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 14:34:32.334560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application compat.raise_( Jul 20 14:34:32.334560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 14:34:32.334560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception Jul 20 14:34:32.334560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 14:34:32.334560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 14:34:32.334560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 14:34:32.334560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 14:34:32.334560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 14:34:32.334560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 14:34:32.335791 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 14:34:32.335791 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 14:34:32.335791 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 20 14:34:32.335791 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 20 14:34:32.335791 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 14:34:32.335791 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 14:34:32.335791 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 14:34:32.335791 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 14:34:32.335791 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 14:34:32.335791 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 14:34:32.335791 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 14:34:32.335791 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 14:34:32.335791 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 14:34:32.335791 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 14:34:32.335791 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 14:34:32.335791 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception Jul 20 14:34:32.336692 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 14:34:32.336692 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 14:34:32.336692 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 14:34:32.336692 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 14:34:32.336692 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 14:34:32.336692 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 14:34:32.336692 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 14:34:32.336692 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application conn.query(q) Jul 20 14:34:32.336692 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 14:34:32.336692 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 14:34:32.336692 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 14:34:32.336692 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application result.read() Jul 20 14:34:32.336692 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 14:34:32.336692 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 14:34:32.336692 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 14:34:32.336692 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 14:34:32.337460 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 14:34:32.337460 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 14:34:32.337460 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 14:34:32.337460 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 14:34:32.337460 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 14:34:32.337460 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 20 14:34:32.337460 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [parameters: {'id': '1b7d8d75672e41ccaa8d3b8f50df4450', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 20 14:34:32.337460 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 14:34:32.337460 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:32.337460 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 14:34:32.337460 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:32.337460 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:32.337460 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:32.337460 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:32.337460 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:32.337460 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:32.338256 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:32.338256 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:32.338256 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:32.338256 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:32.338256 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:32.338256 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:32.338256 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 20 14:34:32.338256 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 20 14:34:32.338256 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:32.338256 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:32.338256 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 20 14:34:32.338256 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 20 14:34:32.338256 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 14:34:32.338256 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 14:34:32.338256 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 20 14:34:32.338256 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:32.339084 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 243/476] 10.222.0.15 () {60 vars in 1131 bytes} [Wed Jul 20 14:34:32 2022] POST /identity/v3/roles => generated 144 bytes in 241 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:32.347174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152e743-4d94-44d2-9a94-49c4978558e7 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:32.407165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d7927-ba53-491c-aa9c-255197db0705 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:32.435481 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:32.446434 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a7349-b164-423d-bc5c-43249b579976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:32.447335 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a7349-b164-423d-bc5c-43249b579976 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:32.447660 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a7349-b164-423d-bc5c-43249b579976 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:32.447945 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a7349-b164-423d-bc5c-43249b579976 None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:32.448774 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a7349-b164-423d-bc5c-43249b579976 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:32.449015 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a7349-b164-423d-bc5c-43249b579976 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:32.450472 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a7349-b164-423d-bc5c-43249b579976 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:32.466632 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 234/477] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:34:32.472466 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a7349-b164-423d-bc5c-43249b579976 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:32.486901 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152e743-4d94-44d2-9a94-49c4978558e7 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:32.524284 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77154eaf-7523-499a-abaa-263d38acb6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:32.525119 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77154eaf-7523-499a-abaa-263d38acb6ab None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:32.525399 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77154eaf-7523-499a-abaa-263d38acb6ab None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:32.525675 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77154eaf-7523-499a-abaa-263d38acb6ab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:32.538765 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77154eaf-7523-499a-abaa-263d38acb6ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:32.548668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-77154eaf-7523-499a-abaa-263d38acb6ab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 14:34:32.548668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:32.548668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:32.548668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:32.548668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:32.548668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:32.548668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:32.548668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:32.548668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:32.548668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:32.548668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:32.548668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:32.548668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:32.548668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:32.548668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:32.548668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:32.550747 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:32.550747 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:32.550747 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:32.550747 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:32.550747 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:32.550747 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:32.550747 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:32.550747 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:32.550747 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:32.550747 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:32.550747 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:32.550747 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:32.550747 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:32.550747 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:32.550747 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:32.550747 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:32.552680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:32.552680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:32.552680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:32.552680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:32.552680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:32.552680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:32.552680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:32.552680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:32.552680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:32.552680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:32.552680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 14:34:32.552680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:32.552680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 244/478] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jul 20 14:34:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:34:32.559451 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77154eaf-7523-499a-abaa-263d38acb6ab None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:32.571382 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:32.591396 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a7349-b164-423d-bc5c-43249b579976 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:32.629360 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea967da4-7b38-4943-a308-5a85f8c411f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:32.630137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea967da4-7b38-4943-a308-5a85f8c411f7 None admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:32.630429 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea967da4-7b38-4943-a308-5a85f8c411f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:32.630709 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea967da4-7b38-4943-a308-5a85f8c411f7 None admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:32.631463 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea967da4-7b38-4943-a308-5a85f8c411f7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:32.633159 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea967da4-7b38-4943-a308-5a85f8c411f7 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:32.656599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 235/479] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jul 20 14:34:32 2022] POST /identity/v3/users => generated 249 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:34:32.672445 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77154eaf-7523-499a-abaa-263d38acb6ab None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:32.692493 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:32.713329 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da67d6d7-8149-4d59-9d9f-70f8b427392a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:32.714187 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67d6d7-8149-4d59-9d9f-70f8b427392a None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:32.714475 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67d6d7-8149-4d59-9d9f-70f8b427392a None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:32.714754 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67d6d7-8149-4d59-9d9f-70f8b427392a None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:32.715591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da67d6d7-8149-4d59-9d9f-70f8b427392a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:32.715969 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da67d6d7-8149-4d59-9d9f-70f8b427392a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:32.717532 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da67d6d7-8149-4d59-9d9f-70f8b427392a None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:32.734251 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 245/480] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:34:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:34:33.999066 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc2d09-0b13-453b-a862-7b234cda348e None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:33.999775 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc2d09-0b13-453b-a862-7b234cda348e None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:33.999775 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc2d09-0b13-453b-a862-7b234cda348e None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:34.001266 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 236/481] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:34.018344 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a7d78-a5e5-4e88-adb0-9679fc6ad576 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:34.018931 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a7d78-a5e5-4e88-adb0-9679fc6ad576 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:34.018931 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a7d78-a5e5-4e88-adb0-9679fc6ad576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:34.039178 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-9e1a7d78-a5e5-4e88-adb0-9679fc6ad576 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:34.040383 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-9e1a7d78-a5e5-4e88-adb0-9679fc6ad576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:34.141757 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 246/482] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:34.181275 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc2d09-0b13-453b-a862-7b234cda348e None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:34.304915 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc2d09-0b13-453b-a862-7b234cda348e None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:34.339907 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384ed80-be75-4204-b387-0296e7b37d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:34.340722 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384ed80-be75-4204-b387-0296e7b37d34 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:34.341007 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384ed80-be75-4204-b387-0296e7b37d34 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:34.341284 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384ed80-be75-4204-b387-0296e7b37d34 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:34.349535 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:34.359388 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384ed80-be75-4204-b387-0296e7b37d34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:34.372455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-4384ed80-be75-4204-b387-0296e7b37d34 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:34:34.372455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:34.372455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:34.372455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:34.372455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:34.372455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:34.372455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:34.372455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:34.372455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:34.372455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:34.372455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:34.372455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:34.372455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:34.372455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:34.372455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:34.372455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:34.374515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:34.374515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:34.374515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:34.374515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:34.374515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:34.374515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:34.374515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:34.374515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:34.374515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:34.374515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:34.374515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:34.374515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:34.374515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:34.374515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:34.374515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:34.374515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:34.376440 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:34.376440 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:34.376440 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:34.376440 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:34.376440 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:34.376440 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:34.376440 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:34.376440 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:34.376440 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:34.376440 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:34.376440 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:34:34.376440 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:34.376440 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 237/483] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jul 20 14:34:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:34.381755 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a7d78-a5e5-4e88-adb0-9679fc6ad576 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:34.543808 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a7d78-a5e5-4e88-adb0-9679fc6ad576 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:34.555307 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5e23c-df3f-43d5-8842-9bcad558c362 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:34.555536 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5e23c-df3f-43d5-8842-9bcad558c362 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:34.555812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5e23c-df3f-43d5-8842-9bcad558c362 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:34.557807 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 238/484] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:34.563682 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a1657-e585-4cd3-a7ca-028027c380fc None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:34.563682 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a1657-e585-4cd3-a7ca-028027c380fc None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:34.564069 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:34.564218 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a1657-e585-4cd3-a7ca-028027c380fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:34.582628 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-218a1657-e585-4cd3-a7ca-028027c380fc None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:34.584472 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-218a1657-e585-4cd3-a7ca-028027c380fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:34.609685 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a823fb-78f0-45a8-8b10-f338f415000e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:34.610432 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a823fb-78f0-45a8-8b10-f338f415000e None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:34.610724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a823fb-78f0-45a8-8b10-f338f415000e None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:34.611009 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a823fb-78f0-45a8-8b10-f338f415000e None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:34.611856 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a823fb-78f0-45a8-8b10-f338f415000e None admin] RBAC: Adding query filter params (name=service) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:34.612173 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a823fb-78f0-45a8-8b10-f338f415000e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:34.613561 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a823fb-78f0-45a8-8b10-f338f415000e None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:34.634480 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 247/485] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jul 20 14:34:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:34.639920 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a823fb-78f0-45a8-8b10-f338f415000e None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:34.659121 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 239/486] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 14:34:34.685599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a1657-e585-4cd3-a7ca-028027c380fc None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:34.835576 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a823fb-78f0-45a8-8b10-f338f415000e None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:34.843583 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a1657-e585-4cd3-a7ca-028027c380fc None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:34.871301 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efab8ee-3720-4ef3-b7a6-e58fb955632b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:34.872512 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efab8ee-3720-4ef3-b7a6-e58fb955632b None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:34.872987 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efab8ee-3720-4ef3-b7a6-e58fb955632b None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:34.873476 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efab8ee-3720-4ef3-b7a6-e58fb955632b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:34.883411 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:34.885827 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efab8ee-3720-4ef3-b7a6-e58fb955632b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:34.887675 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efab8ee-3720-4ef3-b7a6-e58fb955632b None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:34.891365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a5773-8080-4ee4-a9b8-fed6ece37ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:34.892594 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a5773-8080-4ee4-a9b8-fed6ece37ec1 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:34.893063 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a5773-8080-4ee4-a9b8-fed6ece37ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:34.893521 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a5773-8080-4ee4-a9b8-fed6ece37ec1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:34.913832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-7efab8ee-3720-4ef3-b7a6-e58fb955632b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:34.913832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:34.913832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:34.913832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:34.913832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:34.913832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:34.913832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:34.913832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:34.913832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:34.913832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:34.913832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:34.913832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:34.913832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:34.913832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:34.913832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:34.914767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:34.914767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:34.914767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:34.914767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:34.914767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:34.914767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:34.914767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:34.914767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:34.914767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:34.914767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:34.914767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:34.914767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:34.914767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:34.914767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:34.914767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:34.914767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:34.914767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:34.916056 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:34.916056 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:34.916056 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:34.916056 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:34.918132 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 248/487] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:34:34.924310 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304a5773-8080-4ee4-a9b8-fed6ece37ec1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:34.932135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-304a5773-8080-4ee4-a9b8-fed6ece37ec1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:34:34.932135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:34.932135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:34.932135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:34.932135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:34.932135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:34.932135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:34.932135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:34.932135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:34.932135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:34.932135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:34.932135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:34.932135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:34.932135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:34.932135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:34.932135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:34.933109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:34.933109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:34.933109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:34.933109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:34.933109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:34.933109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:34.933109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:34.933109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:34.933109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:34.933109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:34.933109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:34.933109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:34.933109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:34.933109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:34.933109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:34.933109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:34.933891 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:34.933891 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:34.933891 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:34.933891 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:34.933891 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:34.933891 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:34.933891 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:34.933891 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:34.933891 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:34.933891 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:34.933891 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:34:34.933891 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:34.934487 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 240/488] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jul 20 14:34:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:34.935739 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efab8ee-3720-4ef3-b7a6-e58fb955632b None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:34.939769 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a5773-8080-4ee4-a9b8-fed6ece37ec1 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:34.964373 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:35.041934 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efab8ee-3720-4ef3-b7a6-e58fb955632b None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:35.077175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234c440-e2d5-4633-ad2e-84ece28b836e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:35.078127 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234c440-e2d5-4633-ad2e-84ece28b836e None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:35.078313 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a5773-8080-4ee4-a9b8-fed6ece37ec1 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:35.079025 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234c440-e2d5-4633-ad2e-84ece28b836e None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:35.079463 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234c440-e2d5-4633-ad2e-84ece28b836e None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:35.080348 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234c440-e2d5-4633-ad2e-84ece28b836e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:35.080788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234c440-e2d5-4633-ad2e-84ece28b836e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:35.082324 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234c440-e2d5-4633-ad2e-84ece28b836e None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:35.096076 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 249/489] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:35.097712 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f328f-9ce4-4916-ade6-9e21ef0c524c None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:35.098188 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f328f-9ce4-4916-ade6-9e21ef0c524c None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:35.098670 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f328f-9ce4-4916-ade6-9e21ef0c524c None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:35.100166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 250/490] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:35.101340 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f328f-9ce4-4916-ade6-9e21ef0c524c None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:35.115357 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7476951-75ca-4f86-8768-0bd909eeecc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:35.116039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7476951-75ca-4f86-8768-0bd909eeecc6 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:35.116370 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7476951-75ca-4f86-8768-0bd909eeecc6 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:35.116648 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7476951-75ca-4f86-8768-0bd909eeecc6 None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:35.117415 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7476951-75ca-4f86-8768-0bd909eeecc6 None admin] RBAC: Adding query filter params (name=service) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:35.117725 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7476951-75ca-4f86-8768-0bd909eeecc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:35.119359 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7476951-75ca-4f86-8768-0bd909eeecc6 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:35.128890 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 241/491] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jul 20 14:34:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:35.132260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44068ead-84b6-43fa-a65e-c657c0751e64 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:35.132260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44068ead-84b6-43fa-a65e-c657c0751e64 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:35.132260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44068ead-84b6-43fa-a65e-c657c0751e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:35.152937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-44068ead-84b6-43fa-a65e-c657c0751e64 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:35.154156 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-44068ead-84b6-43fa-a65e-c657c0751e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:35.225875 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:35.233019 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:35.255842 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f328f-9ce4-4916-ade6-9e21ef0c524c None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:35.262572 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 242/492] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:35.263685 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44068ead-84b6-43fa-a65e-c657c0751e64 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:35.306571 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42134622-5dd1-43d3-b1f4-e9c434fd6be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:35.307427 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42134622-5dd1-43d3-b1f4-e9c434fd6be9 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:35.307705 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42134622-5dd1-43d3-b1f4-e9c434fd6be9 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:35.307981 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42134622-5dd1-43d3-b1f4-e9c434fd6be9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:35.315636 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42134622-5dd1-43d3-b1f4-e9c434fd6be9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:35.317113 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42134622-5dd1-43d3-b1f4-e9c434fd6be9 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:35.333286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-42134622-5dd1-43d3-b1f4-e9c434fd6be9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 14:34:35.333286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:35.333286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:35.333286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:35.333286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:35.333286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:35.333286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:35.333286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:35.333286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:35.333286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:35.333286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:35.333286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:35.333286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:34:35.333286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:34:35.333286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:34:35.333286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:34:35.334439 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:35.334439 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:35.334439 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:34:35.334439 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:35.334439 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:34:35.334439 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:35.334439 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:35.334439 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:35.334439 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:35.334439 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:35.334439 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:35.334439 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:35.334439 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:35.334439 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:35.334439 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:35.334439 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:35.334439 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:35.334439 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:35.335482 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:35.335482 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:35.335482 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:34:35.335482 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:34:35.335482 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:34:35.335482 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:34:35.335482 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:34:35.335482 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:34:35.335482 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 14:34:35.335482 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:35.339779 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 251/493] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Jul 20 14:34:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:34:35.341228 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d2385-10e9-4053-acc1-77ef292c68ce None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:35.341520 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d2385-10e9-4053-acc1-77ef292c68ce None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:35.341802 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d2385-10e9-4053-acc1-77ef292c68ce None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:35.347544 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 252/494] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:35.348294 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d2385-10e9-4053-acc1-77ef292c68ce None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:35.381671 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44068ead-84b6-43fa-a65e-c657c0751e64 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:35.417161 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2771b74-3237-45cf-833f-e241e22ee8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:35.418087 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2771b74-3237-45cf-833f-e241e22ee8cb None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:35.418470 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2771b74-3237-45cf-833f-e241e22ee8cb None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:35.418804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2771b74-3237-45cf-833f-e241e22ee8cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:35.430293 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2771b74-3237-45cf-833f-e241e22ee8cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:35.431940 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2771b74-3237-45cf-833f-e241e22ee8cb None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:35.451678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-e2771b74-3237-45cf-833f-e241e22ee8cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:35.451678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:35.451678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:35.451678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:35.451678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:35.451678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:35.451678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:35.451678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:35.451678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:35.451678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:35.451678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:35.451678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:35.451678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:35.451678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:35.451678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:35.452564 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:35.452564 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:35.452564 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:35.452564 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:35.452564 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:35.452564 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:35.452564 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:35.452564 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:35.452564 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:35.452564 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:35.452564 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:35.452564 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:35.452564 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:35.452564 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:35.452564 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:35.452564 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:35.452564 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:35.453439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:35.453439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:35.453439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:35.453439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:35.453439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 243/495] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:35.453708 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2771b74-3237-45cf-833f-e241e22ee8cb None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:35.462267 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d2385-10e9-4053-acc1-77ef292c68ce None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:35.503379 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:35.505163 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e609b4e-7293-4784-b777-fa1f20d1b92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:35.506205 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e609b4e-7293-4784-b777-fa1f20d1b92c None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:35.506486 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e609b4e-7293-4784-b777-fa1f20d1b92c None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:35.506827 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e609b4e-7293-4784-b777-fa1f20d1b92c None admin] PATH_INFO: `/v3/roles/service` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:35.517854 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e609b4e-7293-4784-b777-fa1f20d1b92c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:35.525768 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-5e609b4e-7293-4784-b777-fa1f20d1b92c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:34:35.525768 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:35.525768 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:35.525768 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:35.525768 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:35.525768 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:35.525768 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:35.525768 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:35.525768 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:35.525768 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:35.525768 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:35.525768 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:35.525768 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:35.525768 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:35.525768 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:35.525768 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:35.526665 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:35.526665 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:35.526665 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:35.526665 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:35.526665 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:35.526665 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:35.526665 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:35.526665 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:35.526665 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:35.526665 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:35.526665 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:35.526665 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:35.526665 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:35.526665 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:35.526665 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:35.526665 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:35.527524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:35.527524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:35.527524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:35.527524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:35.527524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:35.527524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:35.527524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:35.527524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:35.527524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:35.527524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:35.527524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:34:35.527524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:35.527524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 253/496] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jul 20 14:34:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:35.528343 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8dd79b-f1e0-4141-ad96-1eff608225d1 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:35.528631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8dd79b-f1e0-4141-ad96-1eff608225d1 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:35.528915 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8dd79b-f1e0-4141-ad96-1eff608225d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:35.535342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:35.550425 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-3f8dd79b-f1e0-4141-ad96-1eff608225d1 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:35.551795 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-3f8dd79b-f1e0-4141-ad96-1eff608225d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:35.584586 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2771b74-3237-45cf-833f-e241e22ee8cb None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:35.618505 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa680b30-bf91-4fe1-9ec7-dc1d9a0f7f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:35.619147 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa680b30-bf91-4fe1-9ec7-dc1d9a0f7f72 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:35.619432 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa680b30-bf91-4fe1-9ec7-dc1d9a0f7f72 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:35.619710 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa680b30-bf91-4fe1-9ec7-dc1d9a0f7f72 None admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:35.619843 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 254/497] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:35.620555 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa680b30-bf91-4fe1-9ec7-dc1d9a0f7f72 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:35.620652 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8dd79b-f1e0-4141-ad96-1eff608225d1 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:35.620940 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa680b30-bf91-4fe1-9ec7-dc1d9a0f7f72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:35.622401 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa680b30-bf91-4fe1-9ec7-dc1d9a0f7f72 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:35.660149 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 244/498] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Jul 20 14:34:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:35.662749 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a4ed0-7b60-4e25-b3f4-c879bf8f3133 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:35.663135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a4ed0-7b60-4e25-b3f4-c879bf8f3133 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:35.663524 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a4ed0-7b60-4e25-b3f4-c879bf8f3133 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:35.665128 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 245/499] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:35.665819 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a4ed0-7b60-4e25-b3f4-c879bf8f3133 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:35.720820 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8dd79b-f1e0-4141-ad96-1eff608225d1 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:35.752046 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:35.758394 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead48860-1fa5-4ffc-9e29-c1572cfcdfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:35.759072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead48860-1fa5-4ffc-9e29-c1572cfcdfae None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:35.759387 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead48860-1fa5-4ffc-9e29-c1572cfcdfae None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:35.759666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead48860-1fa5-4ffc-9e29-c1572cfcdfae None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:35.760401 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead48860-1fa5-4ffc-9e29-c1572cfcdfae None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:35.760832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead48860-1fa5-4ffc-9e29-c1572cfcdfae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:35.762118 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead48860-1fa5-4ffc-9e29-c1572cfcdfae None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:35.764013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:35.773729 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 255/500] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:35.773875 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a4ed0-7b60-4e25-b3f4-c879bf8f3133 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:35.774523 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead48860-1fa5-4ffc-9e29-c1572cfcdfae None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:35.810775 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e72fe-bdd1-4c72-adf6-fa4e14d26c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:35.811489 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e72fe-bdd1-4c72-adf6-fa4e14d26c13 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:35.811775 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e72fe-bdd1-4c72-adf6-fa4e14d26c13 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:35.812057 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e72fe-bdd1-4c72-adf6-fa4e14d26c13 None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:35.812838 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07e72fe-bdd1-4c72-adf6-fa4e14d26c13 None admin] RBAC: Adding query filter params (name=service) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:35.813144 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07e72fe-bdd1-4c72-adf6-fa4e14d26c13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:35.814551 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07e72fe-bdd1-4c72-adf6-fa4e14d26c13 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:35.827266 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 246/501] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jul 20 14:34:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:35.827266 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e72fe-bdd1-4c72-adf6-fa4e14d26c13 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:35.878774 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead48860-1fa5-4ffc-9e29-c1572cfcdfae None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:35.914340 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0493e-626a-438a-88c0-78f18bde99db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:35.915183 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0493e-626a-438a-88c0-78f18bde99db None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:35.915480 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0493e-626a-438a-88c0-78f18bde99db None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:35.915750 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0493e-626a-438a-88c0-78f18bde99db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:35.927120 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0493e-626a-438a-88c0-78f18bde99db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:35.928577 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0493e-626a-438a-88c0-78f18bde99db None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:35.932263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e72fe-bdd1-4c72-adf6-fa4e14d26c13 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:35.945058 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-b5d0493e-626a-438a-88c0-78f18bde99db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:35.945058 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:35.945058 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:35.945058 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:35.945058 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:35.945058 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:35.945058 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:35.945058 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:35.945058 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:35.945058 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:35.945058 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:35.945058 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:35.945058 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:35.945058 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:35.945058 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:35.946034 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:35.946034 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:35.946034 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:35.946034 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:35.946034 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:35.946034 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:35.946034 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:35.946034 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:35.946034 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:35.946034 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:35.946034 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:35.946034 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:35.946034 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:35.946034 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:35.946034 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:35.946034 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:35.946034 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:35.946945 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:35.946945 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:35.946945 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:35.946945 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:35.946945 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 256/502] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:35.947601 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205921a8-1446-44e3-a42c-de311c786a04 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:35.947881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205921a8-1446-44e3-a42c-de311c786a04 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:35.948154 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205921a8-1446-44e3-a42c-de311c786a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:35.964382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-205921a8-1446-44e3-a42c-de311c786a04 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:35.965547 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-205921a8-1446-44e3-a42c-de311c786a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:35.972459 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316eb96-8db3-4cce-9edb-023f761ee7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:35.973123 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316eb96-8db3-4cce-9edb-023f761ee7b5 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:35.973412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316eb96-8db3-4cce-9edb-023f761ee7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:35.973693 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316eb96-8db3-4cce-9edb-023f761ee7b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:35.997524 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:36.005316 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:36.033434 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 257/503] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:36.034033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205921a8-1446-44e3-a42c-de311c786a04 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:36.043547 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316eb96-8db3-4cce-9edb-023f761ee7b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:36.045040 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316eb96-8db3-4cce-9edb-023f761ee7b5 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:36.139361 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205921a8-1446-44e3-a42c-de311c786a04 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:36.144558 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 247/504] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:34:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:34:36.145331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316eb96-8db3-4cce-9edb-023f761ee7b5 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:36.173052 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c6457-ec12-4f7c-b867-4aacd4f3621b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:36.174232 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c6457-ec12-4f7c-b867-4aacd4f3621b None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:36.174516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c6457-ec12-4f7c-b867-4aacd4f3621b None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:36.174791 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c6457-ec12-4f7c-b867-4aacd4f3621b None admin] PATH_INFO: `/v3/users/glance` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:36.182534 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c6457-ec12-4f7c-b867-4aacd4f3621b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:36.184040 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c6457-ec12-4f7c-b867-4aacd4f3621b None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:36.200150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-af1c6457-ec12-4f7c-b867-4aacd4f3621b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 14:34:36.200150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:36.200150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:36.200150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:36.200150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:36.200150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:36.200150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:36.200150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:36.200150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:36.200150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:36.200150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:36.200150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:36.200150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:34:36.200150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:34:36.200150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:34:36.200150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:34:36.201254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:36.201254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:36.201254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:34:36.201254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:36.201254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:34:36.201254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:36.201254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:36.201254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:36.201254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:36.201254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:36.201254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:36.201254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:36.201254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:36.201254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:36.201254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:36.201254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:36.201254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:36.201254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:36.203164 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:36.203164 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:36.203164 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:34:36.203164 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:34:36.203164 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:34:36.203164 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:34:36.203164 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:34:36.203164 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:34:36.203164 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 14:34:36.203164 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:36.203704 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:36.203784 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 258/505] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:34:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:36.203784 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c6457-ec12-4f7c-b867-4aacd4f3621b None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:36.237197 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:36.249859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316eb96-8db3-4cce-9edb-023f761ee7b5 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:36.283137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc4a67-0427-4cd2-a47f-bf009a7e12d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:36.283951 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc4a67-0427-4cd2-a47f-bf009a7e12d6 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:36.284228 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc4a67-0427-4cd2-a47f-bf009a7e12d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:36.284496 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc4a67-0427-4cd2-a47f-bf009a7e12d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:36.294728 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dc4a67-0427-4cd2-a47f-bf009a7e12d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:36.296238 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dc4a67-0427-4cd2-a47f-bf009a7e12d6 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:36.302951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c6457-ec12-4f7c-b867-4aacd4f3621b None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:36.313905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-66dc4a67-0427-4cd2-a47f-bf009a7e12d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:36.313905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:36.313905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:36.313905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:36.313905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:36.313905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:36.313905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:36.313905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:36.313905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:36.313905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:36.313905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:36.313905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:36.313905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:36.313905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:36.313905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:36.314750 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:36.314750 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:36.314750 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:36.314750 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:36.314750 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:36.314750 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:36.314750 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:36.314750 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:36.314750 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:36.314750 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:36.314750 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:36.314750 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:36.314750 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:36.314750 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:36.314750 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:36.314750 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:36.314750 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:36.315889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:36.315889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:36.315889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:36.315889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:36.315889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 248/506] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:36.315889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc4a67-0427-4cd2-a47f-bf009a7e12d6 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:36.342026 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810f15b-efd2-4c76-a62a-3cb4a3b50d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:36.342691 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810f15b-efd2-4c76-a62a-3cb4a3b50d6c None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:36.342995 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810f15b-efd2-4c76-a62a-3cb4a3b50d6c None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:36.343287 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810f15b-efd2-4c76-a62a-3cb4a3b50d6c None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:36.344026 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1810f15b-efd2-4c76-a62a-3cb4a3b50d6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:36.344327 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1810f15b-efd2-4c76-a62a-3cb4a3b50d6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:36.345732 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1810f15b-efd2-4c76-a62a-3cb4a3b50d6c None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:36.356925 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 259/507] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:36.357738 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810f15b-efd2-4c76-a62a-3cb4a3b50d6c None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:36.415683 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc4a67-0427-4cd2-a47f-bf009a7e12d6 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:36.422920 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:36.450440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:36.452624 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9092a95-ad65-4aa5-9de7-517cfe422047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:36.453647 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9092a95-ad65-4aa5-9de7-517cfe422047 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:36.453908 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9092a95-ad65-4aa5-9de7-517cfe422047 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:36.454194 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9092a95-ad65-4aa5-9de7-517cfe422047 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:36.457074 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810f15b-efd2-4c76-a62a-3cb4a3b50d6c None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:36.464634 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9092a95-ad65-4aa5-9de7-517cfe422047 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:36.472331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-d9092a95-ad65-4aa5-9de7-517cfe422047 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:34:36.472331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:36.472331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:36.472331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:36.472331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:36.472331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:36.472331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:36.472331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:36.472331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:36.472331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:36.472331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:36.472331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:36.472331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:36.472331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:36.472331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:36.472331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:36.473217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:36.473217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:36.473217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:36.473217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:36.473217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:36.473217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:36.473217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:36.473217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:36.473217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:36.473217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:36.473217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:36.473217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:36.473217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:36.473217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:36.473217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:36.473217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:36.474283 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:36.474283 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:36.474283 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:36.474283 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:36.474283 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:36.474283 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:36.474283 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:36.474283 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:36.474283 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:36.474283 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:36.474283 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:34:36.474283 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:36.474283 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 249/508] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jul 20 14:34:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:36.474283 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9092a95-ad65-4aa5-9de7-517cfe422047 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:36.497380 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff1e6c6-4a10-4991-b2af-341aafb767ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:36.498198 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1e6c6-4a10-4991-b2af-341aafb767ff None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:36.498481 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1e6c6-4a10-4991-b2af-341aafb767ff None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:36.498754 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1e6c6-4a10-4991-b2af-341aafb767ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:36.509631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff1e6c6-4a10-4991-b2af-341aafb767ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:36.510980 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff1e6c6-4a10-4991-b2af-341aafb767ff None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:36.527682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-dff1e6c6-4a10-4991-b2af-341aafb767ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:36.527682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:36.527682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:36.527682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:36.527682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:36.527682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:36.527682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:36.527682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:36.527682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:36.527682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:36.527682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:36.527682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:36.527682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:36.527682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:36.527682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:36.528604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:36.528604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:36.528604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:36.528604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:36.528604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:36.528604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:36.528604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:36.528604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:36.528604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:36.528604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:36.528604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:36.528604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:36.528604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:36.528604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:36.528604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:36.528604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:36.528604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:36.529503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:36.529503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:36.529503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:36.529503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:36.529503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 260/509] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:36.529775 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff1e6c6-4a10-4991-b2af-341aafb767ff None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:36.584949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9092a95-ad65-4aa5-9de7-517cfe422047 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:36.620039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec904033-d58b-456a-84a1-d6bc7dc9f22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:36.620722 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec904033-d58b-456a-84a1-d6bc7dc9f22f None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:36.621002 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec904033-d58b-456a-84a1-d6bc7dc9f22f None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:36.621275 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec904033-d58b-456a-84a1-d6bc7dc9f22f None admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:36.622116 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec904033-d58b-456a-84a1-d6bc7dc9f22f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:36.622424 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec904033-d58b-456a-84a1-d6bc7dc9f22f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:36.624064 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec904033-d58b-456a-84a1-d6bc7dc9f22f None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:36.630675 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff1e6c6-4a10-4991-b2af-341aafb767ff None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:36.642137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 250/510] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Jul 20 14:34:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:36.642927 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec904033-d58b-456a-84a1-d6bc7dc9f22f None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:36.667416 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:36.670665 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767789d-a3a8-4bb2-9909-a1ad24203062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:36.671388 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767789d-a3a8-4bb2-9909-a1ad24203062 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:36.671693 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767789d-a3a8-4bb2-9909-a1ad24203062 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:36.671978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767789d-a3a8-4bb2-9909-a1ad24203062 None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:36.672716 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1767789d-a3a8-4bb2-9909-a1ad24203062 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:36.673023 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1767789d-a3a8-4bb2-9909-a1ad24203062 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:36.674796 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1767789d-a3a8-4bb2-9909-a1ad24203062 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:36.687267 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 261/511] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:36.688091 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767789d-a3a8-4bb2-9909-a1ad24203062 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:36.696661 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:36.754060 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec904033-d58b-456a-84a1-d6bc7dc9f22f None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:36.790000 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c088261-42a5-4765-99e2-1b97cfa54c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:36.790679 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c088261-42a5-4765-99e2-1b97cfa54c72 None admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:36.790957 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c088261-42a5-4765-99e2-1b97cfa54c72 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:36.791261 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c088261-42a5-4765-99e2-1b97cfa54c72 None admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:36.791973 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c088261-42a5-4765-99e2-1b97cfa54c72 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:36.793475 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c088261-42a5-4765-99e2-1b97cfa54c72 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:36.795494 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767789d-a3a8-4bb2-9909-a1ad24203062 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:36.814012 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 251/512] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jul 20 14:34:36 2022] POST /identity/v3/users => generated 247 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:34:36.814832 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c088261-42a5-4765-99e2-1b97cfa54c72 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:36.834028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad928f9e-2b09-4a65-acd3-32d09af49229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:36.834746 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad928f9e-2b09-4a65-acd3-32d09af49229 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:36.835098 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad928f9e-2b09-4a65-acd3-32d09af49229 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:36.835380 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad928f9e-2b09-4a65-acd3-32d09af49229 None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:36.836184 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad928f9e-2b09-4a65-acd3-32d09af49229 None admin] RBAC: Adding query filter params (name=service) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:36.836485 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad928f9e-2b09-4a65-acd3-32d09af49229 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:36.837924 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad928f9e-2b09-4a65-acd3-32d09af49229 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:36.852077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 262/513] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jul 20 14:34:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:36.852077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad928f9e-2b09-4a65-acd3-32d09af49229 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:36.925073 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:36.930888 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:36.956442 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c088261-42a5-4765-99e2-1b97cfa54c72 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:36.966025 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad928f9e-2b09-4a65-acd3-32d09af49229 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:36.994797 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb0781-0b7e-42c4-b1fc-0d3d1d113398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:36.995511 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb0781-0b7e-42c4-b1fc-0d3d1d113398 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:36.995802 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb0781-0b7e-42c4-b1fc-0d3d1d113398 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:36.996082 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb0781-0b7e-42c4-b1fc-0d3d1d113398 None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:36.996821 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb0781-0b7e-42c4-b1fc-0d3d1d113398 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:36.997134 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb0781-0b7e-42c4-b1fc-0d3d1d113398 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:36.998709 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb0781-0b7e-42c4-b1fc-0d3d1d113398 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:37.009109 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145904e-7df5-4686-90fe-d68ce44fdf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:37.010274 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145904e-7df5-4686-90fe-d68ce44fdf8f None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:37.010794 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145904e-7df5-4686-90fe-d68ce44fdf8f None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:37.011277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145904e-7df5-4686-90fe-d68ce44fdf8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:37.012984 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 252/514] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:37.013838 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb0781-0b7e-42c4-b1fc-0d3d1d113398 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:37.088136 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a145904e-7df5-4686-90fe-d68ce44fdf8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:37.089640 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a145904e-7df5-4686-90fe-d68ce44fdf8f None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:37.126301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb0781-0b7e-42c4-b1fc-0d3d1d113398 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:37.160239 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:37.162497 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e1de4-9394-446a-8a7a-c560204a513a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:37.163466 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e1de4-9394-446a-8a7a-c560204a513a None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:37.163750 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e1de4-9394-446a-8a7a-c560204a513a None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:37.164024 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e1de4-9394-446a-8a7a-c560204a513a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:37.171820 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e1de4-9394-446a-8a7a-c560204a513a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:37.173640 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e1de4-9394-446a-8a7a-c560204a513a None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:37.177010 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:37.192168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-a37e1de4-9394-446a-8a7a-c560204a513a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 14:34:37.192168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:37.192168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:37.192168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:37.192168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:37.192168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:37.192168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:37.192168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:37.192168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:37.192168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:37.192168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:37.192168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:37.192168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:34:37.192168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:34:37.192168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:34:37.192168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:34:37.193103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:37.193103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:37.193103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:34:37.193103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:37.193103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:34:37.193103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:37.193103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:37.193103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:37.193103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:37.193103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:37.193103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:37.193103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:37.193103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:37.193103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:37.193103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:37.193103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:37.193103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:37.193103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:37.194009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:37.194009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:37.194009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:34:37.194009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:34:37.194009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:34:37.194009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:34:37.194009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:34:37.194009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:34:37.194009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 14:34:37.194009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:37.194009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 253/515] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:34:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:37.194647 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e1de4-9394-446a-8a7a-c560204a513a None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:37.207772 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 263/516] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:34:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:34:37.207772 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145904e-7df5-4686-90fe-d68ce44fdf8f None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:37.309411 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145904e-7df5-4686-90fe-d68ce44fdf8f None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:37.313054 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e1de4-9394-446a-8a7a-c560204a513a None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:37.343827 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771fcab-1f59-49ea-bcec-fd9489db1206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:37.344619 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771fcab-1f59-49ea-bcec-fd9489db1206 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:37.344902 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771fcab-1f59-49ea-bcec-fd9489db1206 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:37.345191 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771fcab-1f59-49ea-bcec-fd9489db1206 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:37.352200 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f771fcab-1f59-49ea-bcec-fd9489db1206 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:37.352578 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec3a11-8265-496e-becf-9e1f3ba5d66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:37.353727 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec3a11-8265-496e-becf-9e1f3ba5d66b None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:37.353835 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f771fcab-1f59-49ea-bcec-fd9489db1206 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:37.354511 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec3a11-8265-496e-becf-9e1f3ba5d66b None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:37.354973 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec3a11-8265-496e-becf-9e1f3ba5d66b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:37.365760 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dec3a11-8265-496e-becf-9e1f3ba5d66b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:37.367553 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dec3a11-8265-496e-becf-9e1f3ba5d66b None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:37.368051 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-f771fcab-1f59-49ea-bcec-fd9489db1206 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:34:37.368051 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:37.368051 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:37.368051 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:37.368051 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:37.368051 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:37.368051 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:37.368051 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:37.368051 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:37.368051 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:37.368051 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:37.368051 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:37.368051 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:34:37.368051 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:34:37.368051 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:34:37.368882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:34:37.368882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:37.368882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:37.368882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:37.368882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:37.368882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:37.368882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:37.368882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:37.368882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:37.368882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:37.368882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:37.368882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:37.368882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:37.368882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:37.368882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:37.368882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:37.368882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:37.369720 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:34:37.369720 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:34:37.369720 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:34:37.369720 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:34:37.369720 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:34:37.369720 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:34:37.369720 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:34:37.369720 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:37.369720 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 264/517] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:34:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:37.370175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771fcab-1f59-49ea-bcec-fd9489db1206 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:37.392163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-7dec3a11-8265-496e-becf-9e1f3ba5d66b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:37.392163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:37.392163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:37.392163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:37.392163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:37.392163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:37.392163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:37.392163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:37.392163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:37.392163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:37.392163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:37.392163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:37.392163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:37.392163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:37.392163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:37.393081 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:37.393081 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:37.393081 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:37.393081 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:37.393081 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:37.393081 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:37.393081 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:37.393081 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:37.393081 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:37.393081 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:37.393081 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:37.393081 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:37.393081 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:37.393081 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:37.393081 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:37.393081 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:37.393081 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:37.393906 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:37.393906 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:37.393906 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:37.393906 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:37.395968 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 254/518] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:37.396939 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec3a11-8265-496e-becf-9e1f3ba5d66b None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:37.407336 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:37.458059 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:37.501698 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771fcab-1f59-49ea-bcec-fd9489db1206 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:37.511993 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec3a11-8265-496e-becf-9e1f3ba5d66b None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:37.536873 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98bf50-e171-4820-a426-72a499265343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:37.537572 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98bf50-e171-4820-a426-72a499265343 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:37.537858 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98bf50-e171-4820-a426-72a499265343 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:37.538136 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98bf50-e171-4820-a426-72a499265343 None admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:37.539025 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98bf50-e171-4820-a426-72a499265343 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:37.539368 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98bf50-e171-4820-a426-72a499265343 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:37.540983 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98bf50-e171-4820-a426-72a499265343 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:37.548758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09423fcb-a7da-416b-b8da-6e7b067b7789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:37.549562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09423fcb-a7da-416b-b8da-6e7b067b7789 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:37.549843 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09423fcb-a7da-416b-b8da-6e7b067b7789 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:37.550116 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09423fcb-a7da-416b-b8da-6e7b067b7789 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:37.561621 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09423fcb-a7da-416b-b8da-6e7b067b7789 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:37.563202 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09423fcb-a7da-416b-b8da-6e7b067b7789 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:37.584608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 265/519] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Jul 20 14:34:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:37.584608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98bf50-e171-4820-a426-72a499265343 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:37.584844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-09423fcb-a7da-416b-b8da-6e7b067b7789 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:37.584844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:37.584844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:37.584844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:37.584844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:37.584844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:37.584844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:37.584844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:37.584844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:37.584844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:37.584844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:37.584844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:37.584844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:37.584844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:37.584844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:37.585753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:37.585753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:37.585753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:37.585753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:37.585753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:37.585753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:37.585753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:37.585753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:37.585753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:37.585753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:37.585753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:37.585753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:37.585753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:37.585753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:37.585753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:37.585753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:37.585753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:37.586588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:37.586588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:37.586588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:37.586588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:37.586588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 255/520] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:37.586588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09423fcb-a7da-416b-b8da-6e7b067b7789 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:37.677115 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:37.691818 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09423fcb-a7da-416b-b8da-6e7b067b7789 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:37.693094 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98bf50-e171-4820-a426-72a499265343 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:37.695212 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:37.728747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a307a696-d092-4e49-8ba7-0c025dc4ee19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:37.729313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b8706-882e-46c4-92b1-3cbcd384c98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:37.729437 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307a696-d092-4e49-8ba7-0c025dc4ee19 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:37.729611 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307a696-d092-4e49-8ba7-0c025dc4ee19 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:37.729888 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307a696-d092-4e49-8ba7-0c025dc4ee19 None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:37.730192 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b8706-882e-46c4-92b1-3cbcd384c98f None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:37.730548 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b8706-882e-46c4-92b1-3cbcd384c98f None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:37.730634 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a307a696-d092-4e49-8ba7-0c025dc4ee19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:37.730901 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b8706-882e-46c4-92b1-3cbcd384c98f None admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:37.730986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a307a696-d092-4e49-8ba7-0c025dc4ee19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:37.731809 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0b8706-882e-46c4-92b1-3cbcd384c98f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:37.732188 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0b8706-882e-46c4-92b1-3cbcd384c98f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:37.732317 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a307a696-d092-4e49-8ba7-0c025dc4ee19 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:37.733749 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0b8706-882e-46c4-92b1-3cbcd384c98f None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:37.745560 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 256/521] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:37.745560 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a307a696-d092-4e49-8ba7-0c025dc4ee19 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:37.754232 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 266/522] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:34:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:37.755281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b8706-882e-46c4-92b1-3cbcd384c98f None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:37.858824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b8706-882e-46c4-92b1-3cbcd384c98f None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:37.868400 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a307a696-d092-4e49-8ba7-0c025dc4ee19 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:37.897907 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395625a-6bf0-4f9e-8e90-db8b2b81e3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:37.898752 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395625a-6bf0-4f9e-8e90-db8b2b81e3c3 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:37.899042 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395625a-6bf0-4f9e-8e90-db8b2b81e3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:37.899355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395625a-6bf0-4f9e-8e90-db8b2b81e3c3 None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:37.900762 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0395625a-6bf0-4f9e-8e90-db8b2b81e3c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:37.901088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0395625a-6bf0-4f9e-8e90-db8b2b81e3c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:37.902501 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0395625a-6bf0-4f9e-8e90-db8b2b81e3c3 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:37.904158 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dfd483-8056-442b-a138-b5ded7e9e635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:37.904820 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfd483-8056-442b-a138-b5ded7e9e635 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:37.905102 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfd483-8056-442b-a138-b5ded7e9e635 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:37.905375 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfd483-8056-442b-a138-b5ded7e9e635 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:37.918978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 267/523] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:37.918978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395625a-6bf0-4f9e-8e90-db8b2b81e3c3 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:37.928430 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:37.945974 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:37.994471 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dfd483-8056-442b-a138-b5ded7e9e635 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:37.995901 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dfd483-8056-442b-a138-b5ded7e9e635 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:38.025511 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395625a-6bf0-4f9e-8e90-db8b2b81e3c3 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:38.062726 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f0751-9821-4fce-9eb1-b3626fab4651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:38.063863 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f0751-9821-4fce-9eb1-b3626fab4651 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:38.064150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f0751-9821-4fce-9eb1-b3626fab4651 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:38.064427 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f0751-9821-4fce-9eb1-b3626fab4651 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:38.073159 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85f0751-9821-4fce-9eb1-b3626fab4651 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:38.074695 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85f0751-9821-4fce-9eb1-b3626fab4651 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:38.093244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-b85f0751-9821-4fce-9eb1-b3626fab4651 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 14:34:38.093244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:38.093244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:38.093244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:38.093244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:38.093244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:38.093244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:38.093244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:38.093244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:38.093244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:38.093244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:38.093244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:38.093244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:34:38.093244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:34:38.093244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:34:38.093244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:34:38.094446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:38.094446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:38.094446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:34:38.094446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:38.094446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:34:38.094446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:38.094446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:38.094446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:38.094446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:38.094446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:38.094446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:38.094446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:38.094446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:38.094446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:38.094446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:38.094446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:38.094446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:38.094446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:38.095927 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:38.095927 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:38.095927 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:34:38.095927 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:34:38.095927 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:34:38.095927 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:34:38.095927 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:34:38.095927 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:34:38.095927 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 14:34:38.095927 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:38.095927 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 268/524] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jul 20 14:34:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:38.095927 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f0751-9821-4fce-9eb1-b3626fab4651 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:38.108284 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 257/525] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:34:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:34:38.109470 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dfd483-8056-442b-a138-b5ded7e9e635 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:38.169325 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:38.173944 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:38.209082 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f0751-9821-4fce-9eb1-b3626fab4651 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:38.213780 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dfd483-8056-442b-a138-b5ded7e9e635 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:38.247124 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e0f31-facf-4428-9295-eb3f97177c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:38.247715 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e0f31-facf-4428-9295-eb3f97177c65 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:38.247976 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e0f31-facf-4428-9295-eb3f97177c65 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:38.248287 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e0f31-facf-4428-9295-eb3f97177c65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:38.249243 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e0f31-facf-4428-9295-eb3f97177c65 None admin] RBAC: Adding query filter params (role.id=62e9fa78a7c44a16bb4540d3e8d7044d, scope.project.id=59333e4f593346e184877d6bd9d22026, user.id=5d4a107ed04348b59d20d1703dbe04eb) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:38.249365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43652b-4c01-4a34-a092-8c100f04b484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:38.249567 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e0f31-facf-4428-9295-eb3f97177c65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:38.250060 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43652b-4c01-4a34-a092-8c100f04b484 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:38.250341 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43652b-4c01-4a34-a092-8c100f04b484 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:38.250597 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43652b-4c01-4a34-a092-8c100f04b484 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:38.250998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e0f31-facf-4428-9295-eb3f97177c65 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:38.257410 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b43652b-4c01-4a34-a092-8c100f04b484 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:38.258944 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b43652b-4c01-4a34-a092-8c100f04b484 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:38.263410 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 269/526] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:34:38 2022] GET /identity/v3/role_assignments?user.id=5d4a107ed04348b59d20d1703dbe04eb&scope.project.id=59333e4f593346e184877d6bd9d22026&role.id=62e9fa78a7c44a16bb4540d3e8d7044d => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:38.264540 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e0f31-facf-4428-9295-eb3f97177c65 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:38.285655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-8b43652b-4c01-4a34-a092-8c100f04b484 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:34:38.285655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:38.285655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:38.285655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:38.285655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:38.285655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:38.285655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:38.285655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:38.285655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:38.285655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:38.285655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:38.285655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:38.285655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:34:38.285655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:34:38.285655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:34:38.286581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:34:38.286581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:38.286581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:38.286581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:38.286581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:38.286581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:38.286581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:38.286581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:38.286581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:38.286581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:38.286581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:38.286581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:38.286581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:38.286581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:38.286581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:38.286581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:38.286581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:38.287485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:34:38.287485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:34:38.287485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:34:38.287485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:34:38.287485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:34:38.287485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:34:38.287485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:34:38.287485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:38.293813 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 258/527] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:34:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:38.293813 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43652b-4c01-4a34-a092-8c100f04b484 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:38.391599 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e0f31-facf-4428-9295-eb3f97177c65 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:38.425572 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:38.425765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43652b-4c01-4a34-a092-8c100f04b484 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:38.433009 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0905cc-0801-4579-9bd9-4bbe99edfa2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:38.434123 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0905cc-0801-4579-9bd9-4bbe99edfa2a None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:38.434797 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0905cc-0801-4579-9bd9-4bbe99edfa2a None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:38.435157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0905cc-0801-4579-9bd9-4bbe99edfa2a None admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:38.435950 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0905cc-0801-4579-9bd9-4bbe99edfa2a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:38.436258 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0905cc-0801-4579-9bd9-4bbe99edfa2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:38.437694 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0905cc-0801-4579-9bd9-4bbe99edfa2a None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:38.460051 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:38.461125 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 270/528] 10.222.0.15 () {56 vars in 1139 bytes} [Wed Jul 20 14:34:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:38.462112 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0905cc-0801-4579-9bd9-4bbe99edfa2a None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:38.476559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f25fc2-567e-473c-8562-8f79ed396642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:38.477968 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f25fc2-567e-473c-8562-8f79ed396642 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:38.478525 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f25fc2-567e-473c-8562-8f79ed396642 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:38.479168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f25fc2-567e-473c-8562-8f79ed396642 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:38.498569 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f25fc2-567e-473c-8562-8f79ed396642 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:38.500718 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f25fc2-567e-473c-8562-8f79ed396642 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:38.521140 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-75f25fc2-567e-473c-8562-8f79ed396642 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:38.521140 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:38.521140 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:38.521140 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:38.521140 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:38.521140 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:38.521140 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:38.521140 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:38.521140 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:38.521140 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:38.521140 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:38.521140 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:38.521140 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:38.521140 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:38.521140 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:38.522265 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:38.522265 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:38.522265 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:38.522265 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:38.522265 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:38.522265 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:38.522265 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:38.522265 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:38.522265 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:38.522265 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:38.522265 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:38.522265 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:38.522265 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:38.522265 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:38.522265 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:38.522265 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:38.522265 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:38.523346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:38.523346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:38.523346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:38.523346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:38.527412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 259/529] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:38.528513 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f25fc2-567e-473c-8562-8f79ed396642 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:38.603503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0905cc-0801-4579-9bd9-4bbe99edfa2a None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:38.636151 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f25fc2-567e-473c-8562-8f79ed396642 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:38.649072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225816f9-b88a-4fe2-99c4-fa19b951eab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:38.649622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225816f9-b88a-4fe2-99c4-fa19b951eab3 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:38.649905 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225816f9-b88a-4fe2-99c4-fa19b951eab3 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:38.650234 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225816f9-b88a-4fe2-99c4-fa19b951eab3 None admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:38.651193 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225816f9-b88a-4fe2-99c4-fa19b951eab3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:38.651401 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225816f9-b88a-4fe2-99c4-fa19b951eab3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:38.652846 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225816f9-b88a-4fe2-99c4-fa19b951eab3 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:38.669929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 271/530] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:34:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:38.670847 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225816f9-b88a-4fe2-99c4-fa19b951eab3 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:38.679997 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51cfda8-8e0c-4ec7-8117-70debfd52e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:38.681372 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cfda8-8e0c-4ec7-8117-70debfd52e93 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:38.681804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cfda8-8e0c-4ec7-8117-70debfd52e93 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:38.682232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cfda8-8e0c-4ec7-8117-70debfd52e93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:38.726171 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:38.755591 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:38.784700 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51cfda8-8e0c-4ec7-8117-70debfd52e93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:38.786244 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51cfda8-8e0c-4ec7-8117-70debfd52e93 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:38.793655 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225816f9-b88a-4fe2-99c4-fa19b951eab3 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:38.833634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a0e0b-4833-4da5-a42c-f84c861601b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:38.834305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a0e0b-4833-4da5-a42c-f84c861601b6 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:38.834583 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a0e0b-4833-4da5-a42c-f84c861601b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:38.834857 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a0e0b-4833-4da5-a42c-f84c861601b6 None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:38.835667 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452a0e0b-4833-4da5-a42c-f84c861601b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:38.836033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452a0e0b-4833-4da5-a42c-f84c861601b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:38.837779 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452a0e0b-4833-4da5-a42c-f84c861601b6 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:38.854672 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 272/531] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:38.855696 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a0e0b-4833-4da5-a42c-f84c861601b6 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:38.908200 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 260/532] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:34:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:34:38.909378 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51cfda8-8e0c-4ec7-8117-70debfd52e93 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:38.967661 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a0e0b-4833-4da5-a42c-f84c861601b6 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:38.978273 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:38.989761 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:39.006447 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48eaf48-286c-426b-9716-9e1e26b9490b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:39.007094 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48eaf48-286c-426b-9716-9e1e26b9490b None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:39.007380 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48eaf48-286c-426b-9716-9e1e26b9490b None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:39.007663 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48eaf48-286c-426b-9716-9e1e26b9490b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:39.008495 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48eaf48-286c-426b-9716-9e1e26b9490b None admin] RBAC: Adding query filter params (role.id=62e9fa78a7c44a16bb4540d3e8d7044d, scope.project.id=59333e4f593346e184877d6bd9d22026, user.id=15d7d59080cd4832a45dbb6c0349cf4c) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:39.008808 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48eaf48-286c-426b-9716-9e1e26b9490b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:39.010257 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48eaf48-286c-426b-9716-9e1e26b9490b None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:39.013552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51cfda8-8e0c-4ec7-8117-70debfd52e93 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:39.016313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 273/533] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:34:38 2022] GET /identity/v3/role_assignments?user.id=15d7d59080cd4832a45dbb6c0349cf4c&scope.project.id=59333e4f593346e184877d6bd9d22026&role.id=62e9fa78a7c44a16bb4540d3e8d7044d => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:39.025374 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bea34-7807-47a6-bf1d-310697218e10 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:39.025957 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bea34-7807-47a6-bf1d-310697218e10 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:39.026385 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bea34-7807-47a6-bf1d-310697218e10 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:39.028856 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 274/534] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:39 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:39.029873 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0bea34-7807-47a6-bf1d-310697218e10 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:39.069618 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa51223-62e4-4b16-a131-705a813fce6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:39.070681 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa51223-62e4-4b16-a131-705a813fce6b None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:39.070963 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa51223-62e4-4b16-a131-705a813fce6b None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:39.071501 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa51223-62e4-4b16-a131-705a813fce6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:39.081001 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa51223-62e4-4b16-a131-705a813fce6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:39.082564 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa51223-62e4-4b16-a131-705a813fce6b None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:39.098597 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-1aa51223-62e4-4b16-a131-705a813fce6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:34:39.098597 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:39.098597 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:39.098597 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:39.098597 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:39.098597 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:39.098597 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:39.098597 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:39.098597 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:39.098597 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:39.098597 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:39.098597 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:39.098597 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:34:39.098597 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:34:39.098597 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:34:39.099552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:34:39.099552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:39.099552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:39.099552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:39.099552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:39.099552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:39.099552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:39.099552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:39.099552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:39.099552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:39.099552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:39.099552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:39.099552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:39.099552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:39.099552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:39.099552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:39.099552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:39.100406 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:34:39.100406 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:34:39.100406 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:34:39.100406 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:34:39.100406 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:34:39.100406 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:34:39.100406 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:34:39.100406 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:39.104864 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 261/535] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:34:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:39.106324 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78e305-b83b-45a4-a48b-113f7893b529 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:39.106641 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78e305-b83b-45a4-a48b-113f7893b529 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:39.106920 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78e305-b83b-45a4-a48b-113f7893b529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:39.125003 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-ab78e305-b83b-45a4-a48b-113f7893b529 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:39.126403 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-ab78e305-b83b-45a4-a48b-113f7893b529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:39.168099 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0bea34-7807-47a6-bf1d-310697218e10 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:39.201842 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 262/536] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:39.204220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78e305-b83b-45a4-a48b-113f7893b529 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:39.225200 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134220c4-9d18-4bed-8e19-015a96d4959f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:39.226139 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134220c4-9d18-4bed-8e19-015a96d4959f None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:39.227191 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134220c4-9d18-4bed-8e19-015a96d4959f None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:39.227191 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134220c4-9d18-4bed-8e19-015a96d4959f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:39.241685 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134220c4-9d18-4bed-8e19-015a96d4959f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:39.243996 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134220c4-9d18-4bed-8e19-015a96d4959f None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:39.270912 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-134220c4-9d18-4bed-8e19-015a96d4959f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:39.270912 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:39.270912 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:39.270912 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:39.270912 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:39.270912 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:39.270912 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:39.270912 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:39.270912 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:39.270912 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:39.270912 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:39.270912 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:39.270912 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:39.270912 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:39.270912 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:39.271815 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:39.271815 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:39.271815 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:39.271815 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:39.271815 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:39.271815 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:39.271815 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:39.271815 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:39.271815 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:39.271815 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:39.271815 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:39.271815 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:39.271815 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:39.271815 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:39.271815 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:39.271815 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:39.271815 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:39.272999 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:39.272999 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:39.272999 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:39.272999 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:39.275229 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 275/537] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:39.276092 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134220c4-9d18-4bed-8e19-015a96d4959f None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:39.284321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:39.313957 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:39.330367 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78e305-b83b-45a4-a48b-113f7893b529 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:39.368297 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac32861-7faf-4aba-b50b-e619076b0235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:39.369158 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac32861-7faf-4aba-b50b-e619076b0235 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:39.369508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac32861-7faf-4aba-b50b-e619076b0235 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:39.369789 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac32861-7faf-4aba-b50b-e619076b0235 None admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:39.370602 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac32861-7faf-4aba-b50b-e619076b0235 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:39.370916 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac32861-7faf-4aba-b50b-e619076b0235 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:39.372548 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac32861-7faf-4aba-b50b-e619076b0235 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:39.392740 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 263/538] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:34:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:39.393747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac32861-7faf-4aba-b50b-e619076b0235 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:39.410985 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134220c4-9d18-4bed-8e19-015a96d4959f None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:39.460377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b0dd38-bb66-4349-9755-f4f18c0a9a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:39.461332 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0dd38-bb66-4349-9755-f4f18c0a9a54 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:39.461625 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0dd38-bb66-4349-9755-f4f18c0a9a54 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:39.462009 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0dd38-bb66-4349-9755-f4f18c0a9a54 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:39.475061 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b0dd38-bb66-4349-9755-f4f18c0a9a54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:39.483638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-21b0dd38-bb66-4349-9755-f4f18c0a9a54 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:34:39.483638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:39.483638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:39.483638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:39.483638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:39.483638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:39.483638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:39.483638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:39.483638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:39.483638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:39.483638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:39.483638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:39.483638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:39.483638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:39.483638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:39.483638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:39.484559 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:39.484559 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:39.484559 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:39.484559 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:39.484559 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:39.484559 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:39.484559 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:39.484559 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:39.484559 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:39.484559 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:39.484559 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:39.484559 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:39.484559 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:39.484559 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:39.484559 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:39.484559 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:39.485347 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:39.485347 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:39.485347 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:39.485347 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:39.485347 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:39.485347 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:39.485347 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:39.485347 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:39.485347 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:39.485347 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:39.485347 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:34:39.485347 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:39.485347 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 276/539] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jul 20 14:34:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:39.486483 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b0dd38-bb66-4349-9755-f4f18c0a9a54 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:39.511420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac32861-7faf-4aba-b50b-e619076b0235 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:39.538902 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:39.549348 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ca500-47e2-4fbd-9954-de411d841407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:39.550270 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ca500-47e2-4fbd-9954-de411d841407 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:39.550716 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ca500-47e2-4fbd-9954-de411d841407 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:39.551255 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ca500-47e2-4fbd-9954-de411d841407 None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:39.552187 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ca500-47e2-4fbd-9954-de411d841407 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:39.552694 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ca500-47e2-4fbd-9954-de411d841407 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:39.554337 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ca500-47e2-4fbd-9954-de411d841407 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:39.566389 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 264/540] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:39.568498 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ca500-47e2-4fbd-9954-de411d841407 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:39.591440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:39.599682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b0dd38-bb66-4349-9755-f4f18c0a9a54 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:39.638281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f0f66-b8e5-445f-aaca-d6bdd2064604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:39.639059 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f0f66-b8e5-445f-aaca-d6bdd2064604 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:39.639427 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f0f66-b8e5-445f-aaca-d6bdd2064604 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:39.639895 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f0f66-b8e5-445f-aaca-d6bdd2064604 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:39.640744 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15f0f66-b8e5-445f-aaca-d6bdd2064604 None admin] RBAC: Adding query filter params (role.id=62e9fa78a7c44a16bb4540d3e8d7044d, scope.project.id=59333e4f593346e184877d6bd9d22026, user.id=9618ec9bbe76431e84640b0e5ca90eb1) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:39.641171 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15f0f66-b8e5-445f-aaca-d6bdd2064604 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:39.643392 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15f0f66-b8e5-445f-aaca-d6bdd2064604 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:39.649558 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 277/541] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:34:39 2022] GET /identity/v3/role_assignments?user.id=9618ec9bbe76431e84640b0e5ca90eb1&scope.project.id=59333e4f593346e184877d6bd9d22026&role.id=62e9fa78a7c44a16bb4540d3e8d7044d => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:39.650785 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f0f66-b8e5-445f-aaca-d6bdd2064604 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:39.675188 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ca500-47e2-4fbd-9954-de411d841407 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:39.712530 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280dbde1-f9ea-4c69-adf6-96a1bd6e2537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:39.713614 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280dbde1-f9ea-4c69-adf6-96a1bd6e2537 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:39.714098 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280dbde1-f9ea-4c69-adf6-96a1bd6e2537 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:39.714583 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280dbde1-f9ea-4c69-adf6-96a1bd6e2537 None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:39.715588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280dbde1-f9ea-4c69-adf6-96a1bd6e2537 None admin] RBAC: Adding query filter params (name=service) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:39.716134 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280dbde1-f9ea-4c69-adf6-96a1bd6e2537 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:39.717742 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280dbde1-f9ea-4c69-adf6-96a1bd6e2537 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:39.727713 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 265/542] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jul 20 14:34:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:39.735813 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280dbde1-f9ea-4c69-adf6-96a1bd6e2537 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:39.797287 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f0f66-b8e5-445f-aaca-d6bdd2064604 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:39.843812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:39.875170 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a85df-cdb8-4fed-96e0-a76afc2bbef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:39.876102 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a85df-cdb8-4fed-96e0-a76afc2bbef6 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:39.876391 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a85df-cdb8-4fed-96e0-a76afc2bbef6 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:39.876672 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a85df-cdb8-4fed-96e0-a76afc2bbef6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:39.886451 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a85df-cdb8-4fed-96e0-a76afc2bbef6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:39.888012 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a85df-cdb8-4fed-96e0-a76afc2bbef6 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:39.906342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-966a85df-cdb8-4fed-96e0-a76afc2bbef6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:34:39.906342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:39.906342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:39.906342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:39.906342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:39.906342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:39.906342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:39.906342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:39.906342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:39.906342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:39.906342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:39.906342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:39.906342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:34:39.906342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:34:39.906342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:34:39.907440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:34:39.907440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:39.907440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:39.907440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:39.907440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:39.907440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:39.907440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:39.907440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:39.907440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:39.907440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:39.907440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:39.907440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:39.907440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:39.907440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:39.907440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:39.907440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:39.907440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:39.908594 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:34:39.908594 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:34:39.908594 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:34:39.908594 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:34:39.908594 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:34:39.908594 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:34:39.908594 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:34:39.908594 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:39.908594 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 278/543] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:34:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:39.916242 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a85df-cdb8-4fed-96e0-a76afc2bbef6 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:39.920921 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280dbde1-f9ea-4c69-adf6-96a1bd6e2537 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:39.937893 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:39.958347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b7b6d-6816-4653-a7a1-7633d19fc492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:39.959363 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b7b6d-6816-4653-a7a1-7633d19fc492 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:39.959739 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b7b6d-6816-4653-a7a1-7633d19fc492 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:39.960439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b7b6d-6816-4653-a7a1-7633d19fc492 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:39.975323 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b7b6d-6816-4653-a7a1-7633d19fc492 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:39.976992 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b7b6d-6816-4653-a7a1-7633d19fc492 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:39.995746 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-ce9b7b6d-6816-4653-a7a1-7633d19fc492 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:39.995746 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:39.995746 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:39.995746 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:39.995746 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:39.995746 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:39.995746 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:39.995746 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:39.995746 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:39.995746 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:39.995746 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:39.995746 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:39.995746 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:39.995746 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:39.995746 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:39.996836 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:39.996836 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:39.996836 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:39.996836 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:39.996836 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:39.996836 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:39.996836 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:39.996836 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:39.996836 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:39.996836 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:39.996836 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:39.996836 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:39.996836 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:39.996836 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:39.996836 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:39.996836 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:39.996836 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:39.997916 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:39.997916 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:39.997916 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:39.997916 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:39.999601 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 266/544] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:34:40.005129 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b7b6d-6816-4653-a7a1-7633d19fc492 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:40.114127 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a85df-cdb8-4fed-96e0-a76afc2bbef6 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:40.145315 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:40.148672 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b7b6d-6816-4653-a7a1-7633d19fc492 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:40.150072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209aebd1-6711-42ab-a366-8db197d59ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:40.151098 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209aebd1-6711-42ab-a366-8db197d59ec2 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:40.151598 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209aebd1-6711-42ab-a366-8db197d59ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:40.152064 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209aebd1-6711-42ab-a366-8db197d59ec2 None admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:40.155196 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209aebd1-6711-42ab-a366-8db197d59ec2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:40.155764 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209aebd1-6711-42ab-a366-8db197d59ec2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:40.157557 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209aebd1-6711-42ab-a366-8db197d59ec2 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:40.171473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 279/545] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:34:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:40.178035 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209aebd1-6711-42ab-a366-8db197d59ec2 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:40.227191 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96869c-d0b8-4930-a8bf-c650b61cfbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:40.227732 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96869c-d0b8-4930-a8bf-c650b61cfbf1 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:40.227978 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96869c-d0b8-4930-a8bf-c650b61cfbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:40.228217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96869c-d0b8-4930-a8bf-c650b61cfbf1 None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:40.228964 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96869c-d0b8-4930-a8bf-c650b61cfbf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:40.229273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96869c-d0b8-4930-a8bf-c650b61cfbf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:40.230682 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96869c-d0b8-4930-a8bf-c650b61cfbf1 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:40.248863 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:40.257782 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 267/546] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:34:40.262808 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96869c-d0b8-4930-a8bf-c650b61cfbf1 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:40.284798 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209aebd1-6711-42ab-a366-8db197d59ec2 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:40.321981 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b34af92-2ddb-4e11-a6d5-4e033788c6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:40.323007 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34af92-2ddb-4e11-a6d5-4e033788c6d9 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:40.323568 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34af92-2ddb-4e11-a6d5-4e033788c6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:40.324033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34af92-2ddb-4e11-a6d5-4e033788c6d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:40.325062 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b34af92-2ddb-4e11-a6d5-4e033788c6d9 None admin] RBAC: Adding query filter params (role.id=62e9fa78a7c44a16bb4540d3e8d7044d, scope.project.id=59333e4f593346e184877d6bd9d22026, user.id=7658ffde740d446682c1876662815987) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:40.325563 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b34af92-2ddb-4e11-a6d5-4e033788c6d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:40.327223 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b34af92-2ddb-4e11-a6d5-4e033788c6d9 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:40.333654 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 280/547] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:34:40 2022] GET /identity/v3/role_assignments?user.id=7658ffde740d446682c1876662815987&scope.project.id=59333e4f593346e184877d6bd9d22026&role.id=62e9fa78a7c44a16bb4540d3e8d7044d => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:40.447367 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96869c-d0b8-4930-a8bf-c650b61cfbf1 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:40.498068 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca160c-2caf-4c51-b832-effce6c6c1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:40.498955 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca160c-2caf-4c51-b832-effce6c6c1ac None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:40.499482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca160c-2caf-4c51-b832-effce6c6c1ac None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:40.499972 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca160c-2caf-4c51-b832-effce6c6c1ac None admin] PATH_INFO: `/v3/users/swift` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:40.512926 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca160c-2caf-4c51-b832-effce6c6c1ac None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:40.515774 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca160c-2caf-4c51-b832-effce6c6c1ac None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:40.515879 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb190e3f-d766-4cc3-84ac-8ec16de796b8 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:40.516151 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb190e3f-d766-4cc3-84ac-8ec16de796b8 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:40.516431 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb190e3f-d766-4cc3-84ac-8ec16de796b8 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:40.517816 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 281/548] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 14:34:40.522806 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351345ff-3dc6-47d9-bbdc-793c452beb4f None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:40.523699 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351345ff-3dc6-47d9-bbdc-793c452beb4f None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:40.524003 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351345ff-3dc6-47d9-bbdc-793c452beb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:40.539950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-b5ca160c-2caf-4c51-b832-effce6c6c1ac None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 14:34:40.539950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:40.539950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:40.539950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:40.539950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:40.539950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:40.539950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:40.539950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:40.539950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:40.539950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:40.539950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:40.539950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:40.539950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:34:40.539950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:34:40.539950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:34:40.539950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:34:40.541340 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:40.541340 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:40.541340 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:34:40.541340 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:40.541340 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:34:40.541340 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:40.541340 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:40.541340 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:40.541340 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:40.541340 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:40.541340 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:40.541340 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:40.541340 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:40.541340 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:40.541340 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:40.541340 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:40.541340 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:40.541340 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:40.542521 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:40.542521 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:40.542521 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:34:40.542521 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:34:40.542521 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:34:40.542521 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:34:40.542521 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:34:40.542521 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:34:40.542521 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 14:34:40.542521 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:40.543034 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-351345ff-3dc6-47d9-bbdc-793c452beb4f None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:40.543034 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-351345ff-3dc6-47d9-bbdc-793c452beb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:40.545339 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 268/549] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:34:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:34:40.551355 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca160c-2caf-4c51-b832-effce6c6c1ac None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:40.581523 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:40.659348 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 282/550] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 14:34:40.709407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351345ff-3dc6-47d9-bbdc-793c452beb4f None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:40.714898 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca160c-2caf-4c51-b832-effce6c6c1ac None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:40.766245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf18eec-7559-4a6e-aa11-ca29ee4fc641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:40.766928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf18eec-7559-4a6e-aa11-ca29ee4fc641 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:40.767239 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf18eec-7559-4a6e-aa11-ca29ee4fc641 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:40.767520 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf18eec-7559-4a6e-aa11-ca29ee4fc641 None admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:40.768384 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf18eec-7559-4a6e-aa11-ca29ee4fc641 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:40.768813 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf18eec-7559-4a6e-aa11-ca29ee4fc641 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:40.770597 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf18eec-7559-4a6e-aa11-ca29ee4fc641 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:40.791276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 269/551] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Jul 20 14:34:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:34:40.797948 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf18eec-7559-4a6e-aa11-ca29ee4fc641 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:40.839434 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:40.909416 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:40.917700 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351345ff-3dc6-47d9-bbdc-793c452beb4f None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:40.954747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf18eec-7559-4a6e-aa11-ca29ee4fc641 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:40.957243 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557eb95a-d597-45ab-a2a0-40a348041160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:40.958067 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557eb95a-d597-45ab-a2a0-40a348041160 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:40.958351 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557eb95a-d597-45ab-a2a0-40a348041160 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:40.958629 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557eb95a-d597-45ab-a2a0-40a348041160 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:40.970499 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557eb95a-d597-45ab-a2a0-40a348041160 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:40.979206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-557eb95a-d597-45ab-a2a0-40a348041160 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:34:40.979206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:40.979206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:40.979206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:40.979206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:40.979206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:40.979206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:40.979206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:40.979206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:40.979206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:40.979206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:40.979206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:40.979206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:40.979206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:40.979206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:40.979206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:40.980206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:40.980206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:40.980206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:40.980206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:40.980206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:40.980206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:40.980206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:40.980206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:40.980206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:40.980206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:40.980206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:40.980206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:40.980206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:40.980206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:40.980206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:40.980206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:40.981139 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:40.981139 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:40.981139 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:40.981139 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:40.981139 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:40.981139 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:40.981139 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:40.981139 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:40.981139 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:40.981139 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:40.981139 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:34:40.981139 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:40.981139 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 283/552] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jul 20 14:34:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:34:40.988027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557eb95a-d597-45ab-a2a0-40a348041160 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:41.028680 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869f051-2981-4d47-b98a-901b6d358d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:41.029356 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869f051-2981-4d47-b98a-901b6d358d4c None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:41.029639 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869f051-2981-4d47-b98a-901b6d358d4c None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:41.029919 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869f051-2981-4d47-b98a-901b6d358d4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:41.136453 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2869f051-2981-4d47-b98a-901b6d358d4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:41.138298 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2869f051-2981-4d47-b98a-901b6d358d4c None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:41.172518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557eb95a-d597-45ab-a2a0-40a348041160 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:41.196274 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:41.217786 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9adf8a-5581-4515-91d6-89f44ca00449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:41.218482 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9adf8a-5581-4515-91d6-89f44ca00449 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:41.218773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9adf8a-5581-4515-91d6-89f44ca00449 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:41.219076 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9adf8a-5581-4515-91d6-89f44ca00449 None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:41.219856 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9adf8a-5581-4515-91d6-89f44ca00449 None admin] RBAC: Adding query filter params (name=service) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:41.220393 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9adf8a-5581-4515-91d6-89f44ca00449 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:41.221689 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9adf8a-5581-4515-91d6-89f44ca00449 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:41.222235 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:41.240787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 284/553] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jul 20 14:34:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:41.247957 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9adf8a-5581-4515-91d6-89f44ca00449 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:41.252157 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 270/554] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:34:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:34:41.258185 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869f051-2981-4d47-b98a-901b6d358d4c None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:41.364047 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869f051-2981-4d47-b98a-901b6d358d4c None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:41.401079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1420db4c-34f8-4f8b-aa4f-830aa4443619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:41.401880 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420db4c-34f8-4f8b-aa4f-830aa4443619 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:41.402162 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420db4c-34f8-4f8b-aa4f-830aa4443619 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:41.402438 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420db4c-34f8-4f8b-aa4f-830aa4443619 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:41.414513 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1420db4c-34f8-4f8b-aa4f-830aa4443619 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:41.416400 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1420db4c-34f8-4f8b-aa4f-830aa4443619 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:41.434958 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9adf8a-5581-4515-91d6-89f44ca00449 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:41.437827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-1420db4c-34f8-4f8b-aa4f-830aa4443619 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:41.437827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:41.437827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:41.437827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:41.437827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:41.437827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:41.437827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:41.437827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:41.437827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:41.437827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:41.437827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:41.437827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:41.437827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:41.437827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:41.437827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:41.438678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:41.438678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:41.438678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:41.438678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:41.438678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:41.438678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:41.438678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:41.438678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:41.438678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:41.438678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:41.438678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:41.438678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:41.438678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:41.438678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:41.438678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:41.438678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:41.438678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:41.439587 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:41.439587 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:41.439587 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:41.439587 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:41.439587 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 271/555] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:34:41.445211 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42074d6b-e9d3-48a2-9ca2-ede6255db328 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:41.445501 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42074d6b-e9d3-48a2-9ca2-ede6255db328 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:41.445781 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42074d6b-e9d3-48a2-9ca2-ede6255db328 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:41.447872 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 272/556] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:41.449033 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42074d6b-e9d3-48a2-9ca2-ede6255db328 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:41.482511 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:41.485319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37515216-276b-48b2-a648-09e496d92982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:41.486520 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37515216-276b-48b2-a648-09e496d92982 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:41.486832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37515216-276b-48b2-a648-09e496d92982 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:41.487160 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37515216-276b-48b2-a648-09e496d92982 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:41.501396 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37515216-276b-48b2-a648-09e496d92982 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:41.503326 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37515216-276b-48b2-a648-09e496d92982 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:41.526618 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-37515216-276b-48b2-a648-09e496d92982 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:41.526618 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:41.526618 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:41.526618 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:41.526618 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:41.526618 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:41.526618 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:41.526618 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:41.526618 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:41.526618 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:41.526618 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:41.526618 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:41.526618 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:41.526618 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:41.526618 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:41.527551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:41.527551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:41.527551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:41.527551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:41.527551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:41.527551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:41.527551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:41.527551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:41.527551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:41.527551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:41.527551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:41.527551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:41.527551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:41.527551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:41.527551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:41.527551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:41.527551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:41.529002 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:41.529002 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:41.529002 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:41.529002 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:41.531391 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 285/557] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:34:41.532952 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8749676-1094-466e-bcf2-9069827da0e2 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:41.533465 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8749676-1094-466e-bcf2-9069827da0e2 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:41.533931 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8749676-1094-466e-bcf2-9069827da0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:41.552894 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-d8749676-1094-466e-bcf2-9069827da0e2 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:41.554611 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-d8749676-1094-466e-bcf2-9069827da0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:41.584431 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42074d6b-e9d3-48a2-9ca2-ede6255db328 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:41.585429 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:41.629133 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 286/558] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:41.630716 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8749676-1094-466e-bcf2-9069827da0e2 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:41.636701 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fabe79-9c0e-4893-987e-8b1dedfc6bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:41.637743 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fabe79-9c0e-4893-987e-8b1dedfc6bed None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:41.638257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fabe79-9c0e-4893-987e-8b1dedfc6bed None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:41.638739 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fabe79-9c0e-4893-987e-8b1dedfc6bed None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:41.639930 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fabe79-9c0e-4893-987e-8b1dedfc6bed None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:41.640465 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fabe79-9c0e-4893-987e-8b1dedfc6bed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:41.642124 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fabe79-9c0e-4893-987e-8b1dedfc6bed None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:41.668455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 273/559] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:41.668455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fabe79-9c0e-4893-987e-8b1dedfc6bed None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:41.762810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8749676-1094-466e-bcf2-9069827da0e2 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:41.791465 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:41.804410 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a598f1-1ed7-4ba4-92a8-6c2a5e40b8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:41.805274 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fabe79-9c0e-4893-987e-8b1dedfc6bed None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:41.805661 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a598f1-1ed7-4ba4-92a8-6c2a5e40b8a2 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:41.806144 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a598f1-1ed7-4ba4-92a8-6c2a5e40b8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:41.806740 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a598f1-1ed7-4ba4-92a8-6c2a5e40b8a2 None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:41.807818 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a598f1-1ed7-4ba4-92a8-6c2a5e40b8a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:41.808348 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a598f1-1ed7-4ba4-92a8-6c2a5e40b8a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:41.809950 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a598f1-1ed7-4ba4-92a8-6c2a5e40b8a2 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:41.825513 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 287/560] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:41.826715 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a598f1-1ed7-4ba4-92a8-6c2a5e40b8a2 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:41.855010 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401d449-0d97-423a-a966-1abde8dc6568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:41.855855 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401d449-0d97-423a-a966-1abde8dc6568 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:41.856248 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401d449-0d97-423a-a966-1abde8dc6568 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:41.856529 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401d449-0d97-423a-a966-1abde8dc6568 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:41.862584 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:41.876230 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5401d449-0d97-423a-a966-1abde8dc6568 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:41.885956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-5401d449-0d97-423a-a966-1abde8dc6568 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:34:41.885956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:41.885956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:41.885956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:41.885956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:41.885956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:41.885956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:41.885956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:41.885956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:41.885956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:41.885956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:41.885956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:41.885956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:41.885956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:41.885956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:41.885956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:41.886925 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:41.886925 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:41.886925 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:41.886925 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:41.886925 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:41.886925 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:41.886925 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:41.886925 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:41.886925 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:41.886925 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:41.886925 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:41.886925 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:41.886925 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:41.886925 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:41.886925 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:41.886925 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:41.887825 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:41.887825 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:41.887825 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:41.887825 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:41.887825 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:41.887825 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:41.887825 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:41.887825 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:41.887825 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:41.887825 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:41.887825 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:34:41.887825 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:41.891127 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 274/561] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jul 20 14:34:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:41.892377 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401d449-0d97-423a-a966-1abde8dc6568 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:41.970157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a598f1-1ed7-4ba4-92a8-6c2a5e40b8a2 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:42.006319 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401d449-0d97-423a-a966-1abde8dc6568 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:42.011257 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e865b3-9158-43a7-b0b2-3c5dadc4593d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:42.012388 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e865b3-9158-43a7-b0b2-3c5dadc4593d None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:42.012960 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e865b3-9158-43a7-b0b2-3c5dadc4593d None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:42.013479 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e865b3-9158-43a7-b0b2-3c5dadc4593d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:42.021463 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e865b3-9158-43a7-b0b2-3c5dadc4593d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:42.023009 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e865b3-9158-43a7-b0b2-3c5dadc4593d None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:42.037591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-b2e865b3-9158-43a7-b0b2-3c5dadc4593d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:34:42.037591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:42.037591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:42.037591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:42.037591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:42.037591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:42.037591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:42.037591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:42.037591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:42.037591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:42.037591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:42.037591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:42.037591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:34:42.037591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:34:42.037591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:34:42.038615 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:34:42.038615 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:42.038615 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:42.038615 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:42.038615 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:42.038615 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:42.038615 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:42.038615 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:42.038615 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:42.038615 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:42.038615 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:42.038615 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:42.038615 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:42.038615 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:42.038615 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:42.038615 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:42.038615 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:42.039564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:34:42.039564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:34:42.039564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:34:42.039564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:34:42.039564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:34:42.039564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:34:42.039564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:34:42.039564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:42.039564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 288/562] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:34:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:42.040047 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e865b3-9158-43a7-b0b2-3c5dadc4593d None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:42.051830 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14846247-a27f-456d-b9b4-5ee3b81d12f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:42.052901 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14846247-a27f-456d-b9b4-5ee3b81d12f6 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:42.053775 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14846247-a27f-456d-b9b4-5ee3b81d12f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:42.054501 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14846247-a27f-456d-b9b4-5ee3b81d12f6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:42.065695 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14846247-a27f-456d-b9b4-5ee3b81d12f6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:42.068002 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14846247-a27f-456d-b9b4-5ee3b81d12f6 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:42.084873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-14846247-a27f-456d-b9b4-5ee3b81d12f6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 14:34:42.084873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:42.084873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:42.084873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:42.084873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:42.084873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:42.084873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:42.084873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:42.084873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:42.084873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:42.084873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:42.084873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:42.084873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:34:42.084873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:34:42.084873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:34:42.084873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:34:42.086672 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:42.086672 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:42.086672 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:34:42.086672 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:42.086672 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:34:42.086672 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:42.086672 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:42.086672 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:42.086672 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:42.086672 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:42.086672 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:42.086672 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:42.086672 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:42.086672 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:42.086672 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:42.086672 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:42.086672 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:42.086672 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:42.088282 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:42.088282 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:42.088282 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:34:42.088282 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:34:42.088282 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:34:42.088282 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:34:42.088282 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:34:42.088282 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:34:42.088282 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 14:34:42.088282 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:42.088282 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 275/563] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Jul 20 14:34:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:42.088282 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965d44a-094f-491d-8e4f-38c05be223d6 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:42.088282 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965d44a-094f-491d-8e4f-38c05be223d6 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:42.089006 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965d44a-094f-491d-8e4f-38c05be223d6 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:42.089006 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 276/564] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:42.089629 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965d44a-094f-491d-8e4f-38c05be223d6 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:42.103364 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:42.142140 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:42.159367 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e865b3-9158-43a7-b0b2-3c5dadc4593d None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:42.197645 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965d44a-094f-491d-8e4f-38c05be223d6 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:42.198015 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35388dfb-8fb2-4094-bad2-bf4c9e08142c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:42.198728 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35388dfb-8fb2-4094-bad2-bf4c9e08142c None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:42.199016 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35388dfb-8fb2-4094-bad2-bf4c9e08142c None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:42.199432 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35388dfb-8fb2-4094-bad2-bf4c9e08142c None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:42.200313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35388dfb-8fb2-4094-bad2-bf4c9e08142c None admin] RBAC: Adding query filter params (name=service) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:42.200628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35388dfb-8fb2-4094-bad2-bf4c9e08142c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:42.202040 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35388dfb-8fb2-4094-bad2-bf4c9e08142c None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:42.216212 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 289/565] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jul 20 14:34:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:42.217167 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35388dfb-8fb2-4094-bad2-bf4c9e08142c None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:42.233892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec80a4-970c-48e2-abe9-4fa86de244d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:42.234572 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec80a4-970c-48e2-abe9-4fa86de244d6 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:42.234844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec80a4-970c-48e2-abe9-4fa86de244d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:42.235175 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec80a4-970c-48e2-abe9-4fa86de244d6 None admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:42.236169 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec80a4-970c-48e2-abe9-4fa86de244d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:42.236284 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec80a4-970c-48e2-abe9-4fa86de244d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:42.237707 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec80a4-970c-48e2-abe9-4fa86de244d6 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:42.250523 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 277/566] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:34:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:42.251869 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a63c26-bfa7-4dbc-9429-9d9b5a2ef8dc None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:42.252156 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a63c26-bfa7-4dbc-9429-9d9b5a2ef8dc None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:42.252414 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a63c26-bfa7-4dbc-9429-9d9b5a2ef8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:42.274061 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-d3a63c26-bfa7-4dbc-9429-9d9b5a2ef8dc None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:42.275790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-d3a63c26-bfa7-4dbc-9429-9d9b5a2ef8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:42.335316 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35388dfb-8fb2-4094-bad2-bf4c9e08142c None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:42.344315 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:42.359079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 278/567] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:42.359079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a63c26-bfa7-4dbc-9429-9d9b5a2ef8dc None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:42.381605 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e7482-b96e-4b80-b535-6d1308bb2bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:42.382653 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e7482-b96e-4b80-b535-6d1308bb2bbd None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:42.382941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e7482-b96e-4b80-b535-6d1308bb2bbd None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:42.383532 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e7482-b96e-4b80-b535-6d1308bb2bbd None admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:42.384774 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e7482-b96e-4b80-b535-6d1308bb2bbd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:42.384898 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e7482-b96e-4b80-b535-6d1308bb2bbd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:42.386717 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e7482-b96e-4b80-b535-6d1308bb2bbd None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:42.404479 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 290/568] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Jul 20 14:34:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:42.405331 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e7482-b96e-4b80-b535-6d1308bb2bbd None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:42.414176 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:42.468617 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a63c26-bfa7-4dbc-9429-9d9b5a2ef8dc None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:42.509709 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559cf82-6c8f-4fef-be11-c3dcfc342c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:42.510340 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559cf82-6c8f-4fef-be11-c3dcfc342c7f None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:42.510624 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559cf82-6c8f-4fef-be11-c3dcfc342c7f None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:42.511101 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559cf82-6c8f-4fef-be11-c3dcfc342c7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:42.516943 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e7482-b96e-4b80-b535-6d1308bb2bbd None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:42.521641 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0559cf82-6c8f-4fef-be11-c3dcfc342c7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:42.523183 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0559cf82-6c8f-4fef-be11-c3dcfc342c7f None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:42.543181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-0559cf82-6c8f-4fef-be11-c3dcfc342c7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:42.543181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:42.543181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:42.543181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:42.543181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:42.543181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:42.543181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:42.543181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:42.543181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:42.543181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:42.543181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:42.543181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:42.543181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:42.543181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:42.543181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:42.544109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:42.544109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:42.544109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:42.544109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:42.544109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:42.544109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:42.544109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:42.544109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:42.544109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:42.544109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:42.544109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:42.544109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:42.544109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:42.544109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:42.544109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:42.544109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:42.544109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:42.544996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:42.544996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:42.544996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:42.544996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:42.544996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 279/569] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:42.545283 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559cf82-6c8f-4fef-be11-c3dcfc342c7f None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:42.558411 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f381f-8072-4124-91ed-d47d0ff413a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:42.559114 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f381f-8072-4124-91ed-d47d0ff413a8 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:42.559398 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f381f-8072-4124-91ed-d47d0ff413a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:42.559673 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f381f-8072-4124-91ed-d47d0ff413a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:42.560532 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26f381f-8072-4124-91ed-d47d0ff413a8 None admin] RBAC: Adding query filter params (role.id=62e9fa78a7c44a16bb4540d3e8d7044d, scope.project.id=59333e4f593346e184877d6bd9d22026, user.id=37f4844988474d7a91133d77906611a5) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:42.560848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26f381f-8072-4124-91ed-d47d0ff413a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:42.562329 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26f381f-8072-4124-91ed-d47d0ff413a8 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:42.568459 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 291/570] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:34:42 2022] GET /identity/v3/role_assignments?user.id=37f4844988474d7a91133d77906611a5&scope.project.id=59333e4f593346e184877d6bd9d22026&role.id=62e9fa78a7c44a16bb4540d3e8d7044d => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:42.570348 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f381f-8072-4124-91ed-d47d0ff413a8 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:42.605959 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:42.659085 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559cf82-6c8f-4fef-be11-c3dcfc342c7f None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:42.675402 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:42.697399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f8490-6c05-4e5a-bc88-055b1f06f81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:42.697864 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f8490-6c05-4e5a-bc88-055b1f06f81e None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:42.698145 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f8490-6c05-4e5a-bc88-055b1f06f81e None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:42.698416 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f8490-6c05-4e5a-bc88-055b1f06f81e None admin] PATH_INFO: `/v3/roles/service` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:42.710227 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9f8490-6c05-4e5a-bc88-055b1f06f81e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:42.712112 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f381f-8072-4124-91ed-d47d0ff413a8 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:42.718321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-9e9f8490-6c05-4e5a-bc88-055b1f06f81e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:34:42.718321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:42.718321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:42.718321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:42.718321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:42.718321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:42.718321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:42.718321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:42.718321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:42.718321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:42.718321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:42.718321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:42.718321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:42.718321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:42.718321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:42.718321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:42.719365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:42.719365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:42.719365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:42.719365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:42.719365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:42.719365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:42.719365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:42.719365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:42.719365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:42.719365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:42.719365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:42.719365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:42.719365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:42.719365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:42.719365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:42.719365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:42.720219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:42.720219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:42.720219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:42.720219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:42.720219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:42.720219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:42.720219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:42.720219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:42.720219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:42.720219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:42.720219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:34:42.720219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:42.720219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 280/571] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jul 20 14:34:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:42.721010 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f8490-6c05-4e5a-bc88-055b1f06f81e None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:42.755188 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c3a18d-6ffa-4cea-a018-807d23ff38d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:42.756003 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3a18d-6ffa-4cea-a018-807d23ff38d9 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:42.756935 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3a18d-6ffa-4cea-a018-807d23ff38d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:42.756935 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3a18d-6ffa-4cea-a018-807d23ff38d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:42.768789 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c3a18d-6ffa-4cea-a018-807d23ff38d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:42.770919 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c3a18d-6ffa-4cea-a018-807d23ff38d9 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:42.789195 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-f8c3a18d-6ffa-4cea-a018-807d23ff38d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:42.789195 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:42.789195 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:42.789195 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:42.789195 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:42.789195 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:42.789195 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:42.789195 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:42.789195 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:42.789195 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:42.789195 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:42.789195 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:42.789195 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:42.789195 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:42.789195 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:42.790319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:42.790319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:42.790319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:42.790319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:42.790319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:42.790319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:42.790319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:42.790319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:42.790319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:42.790319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:42.790319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:42.790319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:42.790319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:42.790319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:42.790319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:42.790319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:42.790319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:42.791653 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:42.791653 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:42.791653 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:42.791653 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:42.791653 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 292/572] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:42.791970 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fcb80-7067-49ad-919a-0ce211d1e3d1 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:42.792600 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fcb80-7067-49ad-919a-0ce211d1e3d1 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:42.793106 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fcb80-7067-49ad-919a-0ce211d1e3d1 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:42.799906 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 293/573] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:42.799906 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fcb80-7067-49ad-919a-0ce211d1e3d1 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:42.841990 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f8490-6c05-4e5a-bc88-055b1f06f81e None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:42.855544 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:42.877894 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40d3cc-9d97-4bf1-b826-ec30ff5ee063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:42.878562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40d3cc-9d97-4bf1-b826-ec30ff5ee063 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:42.878845 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40d3cc-9d97-4bf1-b826-ec30ff5ee063 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:42.879141 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40d3cc-9d97-4bf1-b826-ec30ff5ee063 None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:42.879869 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c40d3cc-9d97-4bf1-b826-ec30ff5ee063 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:42.880413 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c40d3cc-9d97-4bf1-b826-ec30ff5ee063 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:42.881716 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c40d3cc-9d97-4bf1-b826-ec30ff5ee063 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:42.895612 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 281/574] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:42.896456 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40d3cc-9d97-4bf1-b826-ec30ff5ee063 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:42.910773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fcb80-7067-49ad-919a-0ce211d1e3d1 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:42.937454 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:42.951321 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89135d2f-fe4d-4ec4-832b-51f9bead3895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:42.952150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89135d2f-fe4d-4ec4-832b-51f9bead3895 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:42.952818 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89135d2f-fe4d-4ec4-832b-51f9bead3895 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:42.952818 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89135d2f-fe4d-4ec4-832b-51f9bead3895 None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:42.953752 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89135d2f-fe4d-4ec4-832b-51f9bead3895 None admin] RBAC: Adding query filter params (name=service) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:42.954151 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89135d2f-fe4d-4ec4-832b-51f9bead3895 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:42.955666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89135d2f-fe4d-4ec4-832b-51f9bead3895 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:42.969367 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 294/575] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jul 20 14:34:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:42.969367 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867cda9-e394-48b6-9191-dec21fe0e3a9 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:42.969367 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867cda9-e394-48b6-9191-dec21fe0e3a9 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:42.969367 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867cda9-e394-48b6-9191-dec21fe0e3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:42.984113 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-9867cda9-e394-48b6-9191-dec21fe0e3a9 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:42.985327 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-9867cda9-e394-48b6-9191-dec21fe0e3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:43.006505 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40d3cc-9d97-4bf1-b826-ec30ff5ee063 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:43.046832 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54d095-5c75-4b8c-843d-ea77ac3b08fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:43.047798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54d095-5c75-4b8c-843d-ea77ac3b08fe None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:43.048629 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54d095-5c75-4b8c-843d-ea77ac3b08fe None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:43.049001 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54d095-5c75-4b8c-843d-ea77ac3b08fe None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:43.049833 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa54d095-5c75-4b8c-843d-ea77ac3b08fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:43.050291 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa54d095-5c75-4b8c-843d-ea77ac3b08fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:43.051859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa54d095-5c75-4b8c-843d-ea77ac3b08fe None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:43.063294 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 295/576] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:43.063294 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9867cda9-e394-48b6-9191-dec21fe0e3a9 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:43.075770 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 282/577] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:43.076709 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54d095-5c75-4b8c-843d-ea77ac3b08fe None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:43.109183 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:43.170150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9867cda9-e394-48b6-9191-dec21fe0e3a9 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:43.185773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:43.188846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54d095-5c75-4b8c-843d-ea77ac3b08fe None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:43.209297 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f659fe97-9218-4da9-b802-95646c2106cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:43.210518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659fe97-9218-4da9-b802-95646c2106cf None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:43.210949 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659fe97-9218-4da9-b802-95646c2106cf None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:43.211487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659fe97-9218-4da9-b802-95646c2106cf None admin] PATH_INFO: `/v3/users/glance` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:43.219502 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f659fe97-9218-4da9-b802-95646c2106cf None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:43.221157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f659fe97-9218-4da9-b802-95646c2106cf None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:43.222899 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2896fa-7965-4ba4-a09c-d854644212d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:43.223746 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2896fa-7965-4ba4-a09c-d854644212d2 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:43.224019 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2896fa-7965-4ba4-a09c-d854644212d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:43.224291 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2896fa-7965-4ba4-a09c-d854644212d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:43.235106 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2896fa-7965-4ba4-a09c-d854644212d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:43.236589 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2896fa-7965-4ba4-a09c-d854644212d2 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:43.237166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-f659fe97-9218-4da9-b802-95646c2106cf None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 14:34:43.237166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:43.237166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:43.237166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:43.237166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:43.237166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:43.237166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:43.237166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:43.237166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:43.237166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:43.237166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:43.237166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:43.237166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:34:43.237166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:34:43.237166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:34:43.237166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:34:43.238266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:43.238266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:43.238266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:34:43.238266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:43.238266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:34:43.238266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:43.238266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:43.238266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:43.238266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:43.238266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:43.238266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:43.238266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:43.238266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:43.238266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:43.238266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:43.238266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:43.238266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:43.238266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:43.239247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:43.239247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:43.239247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:34:43.239247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:34:43.239247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:34:43.239247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:34:43.239247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:34:43.239247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:34:43.239247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 14:34:43.239247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:43.239247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 296/578] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:34:43 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:43.239247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f659fe97-9218-4da9-b802-95646c2106cf None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:43.261079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-2c2896fa-7965-4ba4-a09c-d854644212d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:43.261079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:43.261079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:43.261079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:43.261079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:43.261079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:43.261079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:43.261079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:43.261079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:43.261079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:43.261079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:43.261079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:43.261079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:43.261079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:43.261079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:43.262349 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:43.262349 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:43.262349 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:43.262349 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:43.262349 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:43.262349 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:43.262349 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:43.262349 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:43.262349 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:43.262349 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:43.262349 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:43.262349 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:43.262349 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:43.262349 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:43.262349 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:43.262349 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:43.262349 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:43.263728 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:43.263728 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:43.263728 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:43.263728 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:43.263728 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 283/579] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:43.264003 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2896fa-7965-4ba4-a09c-d854644212d2 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:43.357332 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f659fe97-9218-4da9-b802-95646c2106cf None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:43.359924 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:43.377134 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2896fa-7965-4ba4-a09c-d854644212d2 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:43.393378 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a24c6-834b-4a66-847f-1f6fce8b6151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:43.394191 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a24c6-834b-4a66-847f-1f6fce8b6151 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:43.394473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a24c6-834b-4a66-847f-1f6fce8b6151 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:43.394762 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a24c6-834b-4a66-847f-1f6fce8b6151 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:43.405369 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302a24c6-834b-4a66-847f-1f6fce8b6151 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:43.413236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-302a24c6-834b-4a66-847f-1f6fce8b6151 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:34:43.413236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:43.413236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:43.413236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:43.413236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:43.413236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:43.413236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:43.413236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:43.413236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:43.413236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:43.413236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:43.413236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:43.413236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:43.413236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:43.413236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:43.413236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:43.414167 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:43.414167 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:43.414167 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:43.414167 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:43.414167 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:43.414167 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:43.414167 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:43.414167 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:43.414167 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:43.414167 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:43.414167 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:43.414167 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:43.414167 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:43.414167 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:43.414167 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:43.414167 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:43.414992 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:43.414992 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:43.414992 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:43.414992 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:43.414992 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:43.414992 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:43.414992 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:43.414992 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:43.414992 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:43.414992 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:43.414992 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:34:43.414992 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:43.415665 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fcdf08-8950-425f-b7d9-d48139267791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:43.415665 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fcdf08-8950-425f-b7d9-d48139267791 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:43.415921 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 297/580] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jul 20 14:34:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:43.415921 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a24c6-834b-4a66-847f-1f6fce8b6151 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:43.420221 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fcdf08-8950-425f-b7d9-d48139267791 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:43.420221 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fcdf08-8950-425f-b7d9-d48139267791 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:43.427814 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fcdf08-8950-425f-b7d9-d48139267791 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:43.429332 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fcdf08-8950-425f-b7d9-d48139267791 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:43.443804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-19fcdf08-8950-425f-b7d9-d48139267791 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:34:43.443804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:43.443804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:43.443804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:43.443804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:43.443804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:43.443804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:43.443804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:43.443804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:43.443804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:43.443804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:43.443804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:43.443804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:34:43.443804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:34:43.443804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:34:43.444928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:34:43.444928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:43.444928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:43.444928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:43.444928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:43.444928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:43.444928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:43.444928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:43.444928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:43.444928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:43.444928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:43.444928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:43.444928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:43.444928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:43.444928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:43.444928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:43.444928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:43.445925 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:34:43.445925 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:34:43.445925 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:34:43.445925 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:34:43.445925 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:34:43.445925 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:34:43.445925 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:34:43.445925 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:43.445925 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 284/581] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:34:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:43.446496 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fcdf08-8950-425f-b7d9-d48139267791 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:43.458584 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:43.535020 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a24c6-834b-4a66-847f-1f6fce8b6151 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:43.556076 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fcdf08-8950-425f-b7d9-d48139267791 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:43.571945 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11018b-aeb4-4885-ada3-4ac85b8343ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:43.572772 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11018b-aeb4-4885-ada3-4ac85b8343ca None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:43.573166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11018b-aeb4-4885-ada3-4ac85b8343ca None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:43.573528 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11018b-aeb4-4885-ada3-4ac85b8343ca None admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:43.574476 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11018b-aeb4-4885-ada3-4ac85b8343ca None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:43.574910 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11018b-aeb4-4885-ada3-4ac85b8343ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:43.576509 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11018b-aeb4-4885-ada3-4ac85b8343ca None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:43.592623 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d6078-000f-4b6e-9e28-8e2add2bb151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:43.593314 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d6078-000f-4b6e-9e28-8e2add2bb151 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:43.593591 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d6078-000f-4b6e-9e28-8e2add2bb151 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:43.593865 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d6078-000f-4b6e-9e28-8e2add2bb151 None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:43.594574 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365d6078-000f-4b6e-9e28-8e2add2bb151 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:43.594959 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365d6078-000f-4b6e-9e28-8e2add2bb151 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:43.596464 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365d6078-000f-4b6e-9e28-8e2add2bb151 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:43.604048 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 298/582] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Jul 20 14:34:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:43.604048 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11018b-aeb4-4885-ada3-4ac85b8343ca None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:43.619254 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 285/583] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:43.620065 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d6078-000f-4b6e-9e28-8e2add2bb151 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:43.624714 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:43.706742 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:43.716290 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11018b-aeb4-4885-ada3-4ac85b8343ca None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:43.740151 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d6078-000f-4b6e-9e28-8e2add2bb151 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:43.752908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c3194-48f1-40db-9536-547397f7e512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:43.753749 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c3194-48f1-40db-9536-547397f7e512 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:43.754134 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c3194-48f1-40db-9536-547397f7e512 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:43.754782 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c3194-48f1-40db-9536-547397f7e512 None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:43.755378 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c3194-48f1-40db-9536-547397f7e512 None admin] RBAC: Adding query filter params (name=service) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:43.755748 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c3194-48f1-40db-9536-547397f7e512 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:43.757252 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c3194-48f1-40db-9536-547397f7e512 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:43.770065 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 299/584] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jul 20 14:34:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:43.770065 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c3194-48f1-40db-9536-547397f7e512 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:43.780840 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59c955-0227-4937-988b-3efcf74571bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:43.781644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59c955-0227-4937-988b-3efcf74571bf None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:43.782042 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59c955-0227-4937-988b-3efcf74571bf None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:43.782394 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59c955-0227-4937-988b-3efcf74571bf None admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:43.783294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab59c955-0227-4937-988b-3efcf74571bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:43.783674 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab59c955-0227-4937-988b-3efcf74571bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:43.785212 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab59c955-0227-4937-988b-3efcf74571bf None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:43.797846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 286/585] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:34:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:43.798913 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59c955-0227-4937-988b-3efcf74571bf None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:43.867445 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:43.879033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c3194-48f1-40db-9536-547397f7e512 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:43.915350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59c955-0227-4937-988b-3efcf74571bf None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:43.924010 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31000f1-f434-4651-ad48-3c84c4080b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:43.924832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31000f1-f434-4651-ad48-3c84c4080b57 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:43.925119 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31000f1-f434-4651-ad48-3c84c4080b57 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:43.925395 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31000f1-f434-4651-ad48-3c84c4080b57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:43.937035 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31000f1-f434-4651-ad48-3c84c4080b57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:43.938516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31000f1-f434-4651-ad48-3c84c4080b57 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:43.953495 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a467f2a-80f1-45d7-8880-f561def3d76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:43.954297 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a467f2a-80f1-45d7-8880-f561def3d76b None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:43.954572 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a467f2a-80f1-45d7-8880-f561def3d76b None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:43.954864 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a467f2a-80f1-45d7-8880-f561def3d76b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:43.957583 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-e31000f1-f434-4651-ad48-3c84c4080b57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:43.957583 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:43.957583 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:43.957583 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:43.957583 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:43.957583 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:43.957583 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:43.957583 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:43.957583 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:43.957583 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:43.957583 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:43.957583 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:43.957583 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:43.957583 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:43.957583 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:43.958622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:43.958622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:43.958622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:43.958622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:43.958622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:43.958622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:43.958622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:43.958622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:43.958622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:43.958622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:43.958622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:43.958622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:43.958622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:43.958622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:43.958622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:43.958622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:43.958622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:43.959554 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:43.959554 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:43.959554 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:43.959554 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:43.959554 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 300/586] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:43.959869 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31000f1-f434-4651-ad48-3c84c4080b57 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:43.965347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a467f2a-80f1-45d7-8880-f561def3d76b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:43.967023 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a467f2a-80f1-45d7-8880-f561def3d76b None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:43.973959 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:43.983882 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-7a467f2a-80f1-45d7-8880-f561def3d76b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 14:34:43.983882 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:43.983882 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:43.983882 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:43.983882 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:43.983882 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:43.983882 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:43.983882 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:43.983882 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:43.983882 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:43.983882 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:43.983882 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:43.983882 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:34:43.983882 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:34:43.983882 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:34:43.983882 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:34:43.984998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:43.984998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:43.984998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:34:43.984998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:43.984998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:34:43.984998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:43.984998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:43.984998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:43.984998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:43.984998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:43.984998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:43.984998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:43.984998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:43.984998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:43.984998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:43.984998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:43.984998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:43.984998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:43.986136 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:43.986136 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:43.986136 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:34:43.986136 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:34:43.986136 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:34:43.986136 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:34:43.986136 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:34:43.986136 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:34:43.986136 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 14:34:43.986136 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:43.987225 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 287/587] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:34:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:43.987864 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a467f2a-80f1-45d7-8880-f561def3d76b None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:44.084874 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31000f1-f434-4651-ad48-3c84c4080b57 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:44.092400 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a467f2a-80f1-45d7-8880-f561def3d76b None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:44.118005 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:44.120978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a79045-fc27-48e0-afdf-3ae8b0553208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:44.121935 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a79045-fc27-48e0-afdf-3ae8b0553208 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:44.122325 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a79045-fc27-48e0-afdf-3ae8b0553208 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:44.122708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a79045-fc27-48e0-afdf-3ae8b0553208 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:44.130402 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2f670-3c58-49be-9c9a-1531c694a6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:44.131278 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2f670-3c58-49be-9c9a-1531c694a6bb None admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:44.131562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2f670-3c58-49be-9c9a-1531c694a6bb None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:44.131900 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2f670-3c58-49be-9c9a-1531c694a6bb None admin] PATH_INFO: `/v3/projects/59333e4f593346e184877d6bd9d22026/users/5d4a107ed04348b59d20d1703dbe04eb/roles/62e9fa78a7c44a16bb4540d3e8d7044d` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:44.134492 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a79045-fc27-48e0-afdf-3ae8b0553208 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:44.136102 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a79045-fc27-48e0-afdf-3ae8b0553208 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:44.147723 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff2f670-3c58-49be-9c9a-1531c694a6bb None admin] RBAC: Authorizing `identity:create_grant(project_id=59333e4f593346e184877d6bd9d22026, user_id=5d4a107ed04348b59d20d1703dbe04eb, role_id=62e9fa78a7c44a16bb4540d3e8d7044d)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:44.149183 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff2f670-3c58-49be-9c9a-1531c694a6bb None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:44.154307 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-58a79045-fc27-48e0-afdf-3ae8b0553208 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:44.154307 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:44.154307 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:44.154307 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:44.154307 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:44.154307 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:44.154307 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:44.154307 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:44.154307 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:44.154307 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:44.154307 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:44.154307 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:44.154307 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:44.154307 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:44.154307 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:44.160449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:44.160449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:44.160449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:44.160449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:44.160449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:44.160449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:44.160449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:44.160449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:44.160449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:44.160449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:44.160449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:44.160449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:44.160449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:44.160449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:44.160449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:44.160449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:44.160449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:44.161642 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:44.161642 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:44.161642 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:44.161642 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:44.161642 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 301/588] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:44.161642 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a79045-fc27-48e0-afdf-3ae8b0553208 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:44.179268 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 288/589] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jul 20 14:34:43 2022] PUT /identity/v3/projects/59333e4f593346e184877d6bd9d22026/users/5d4a107ed04348b59d20d1703dbe04eb/roles/62e9fa78a7c44a16bb4540d3e8d7044d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:34:44.179268 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2f670-3c58-49be-9c9a-1531c694a6bb None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:44.249750 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:44.307257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2f670-3c58-49be-9c9a-1531c694a6bb None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:44.323160 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a79045-fc27-48e0-afdf-3ae8b0553208 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:44.355537 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ebcdf-62d7-48f5-ac7c-9bcb07263449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:44.356510 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ebcdf-62d7-48f5-ac7c-9bcb07263449 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:44.356802 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ebcdf-62d7-48f5-ac7c-9bcb07263449 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:44.357078 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ebcdf-62d7-48f5-ac7c-9bcb07263449 None admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:44.358096 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4ebcdf-62d7-48f5-ac7c-9bcb07263449 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:44.358414 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4ebcdf-62d7-48f5-ac7c-9bcb07263449 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:44.360758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4ebcdf-62d7-48f5-ac7c-9bcb07263449 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:44.369502 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c1374a-f101-443f-8ce6-d3a9fe26c51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:44.370182 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1374a-f101-443f-8ce6-d3a9fe26c51c None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:44.370464 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1374a-f101-443f-8ce6-d3a9fe26c51c None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:44.370738 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1374a-f101-443f-8ce6-d3a9fe26c51c None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:44.374665 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c1374a-f101-443f-8ce6-d3a9fe26c51c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:44.374665 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c1374a-f101-443f-8ce6-d3a9fe26c51c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:44.374665 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c1374a-f101-443f-8ce6-d3a9fe26c51c None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:44.384918 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 289/590] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Jul 20 14:34:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:44.385836 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ebcdf-62d7-48f5-ac7c-9bcb07263449 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:44.387310 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 302/591] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:44.390715 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c1374a-f101-443f-8ce6-d3a9fe26c51c None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:44.424887 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:44.506254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c1374a-f101-443f-8ce6-d3a9fe26c51c None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:44.522323 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ebcdf-62d7-48f5-ac7c-9bcb07263449 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:44.541235 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41cf8b-7dd9-4c95-a0f5-2027d70a82fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:44.542247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41cf8b-7dd9-4c95-a0f5-2027d70a82fa None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:44.542632 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41cf8b-7dd9-4c95-a0f5-2027d70a82fa None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:44.543077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41cf8b-7dd9-4c95-a0f5-2027d70a82fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:44.553762 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f41cf8b-7dd9-4c95-a0f5-2027d70a82fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:44.554604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:44.559748 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f41cf8b-7dd9-4c95-a0f5-2027d70a82fa None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:44.563675 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056be8aa-7274-4e09-9503-9071987aa3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:44.564357 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056be8aa-7274-4e09-9503-9071987aa3d0 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:44.564638 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056be8aa-7274-4e09-9503-9071987aa3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:44.564919 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056be8aa-7274-4e09-9503-9071987aa3d0 None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:44.565944 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056be8aa-7274-4e09-9503-9071987aa3d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:44.566254 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056be8aa-7274-4e09-9503-9071987aa3d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:44.567678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056be8aa-7274-4e09-9503-9071987aa3d0 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:44.581639 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-4f41cf8b-7dd9-4c95-a0f5-2027d70a82fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:44.581639 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:44.581639 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:44.581639 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:44.581639 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:44.581639 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:44.581639 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:44.581639 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:44.581639 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:44.581639 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:44.581639 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:44.581639 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:44.581639 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:44.581639 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:44.581639 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:44.582866 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:44.582866 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:44.582866 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:44.582866 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:44.582866 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:44.582866 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:44.582866 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:44.582866 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:44.582866 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:44.582866 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:44.582866 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:44.582866 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:44.582866 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:44.582866 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:44.582866 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:44.582866 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:44.582866 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:44.583998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:44.583998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:44.583998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:44.583998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:44.585677 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 303/593] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:34:44.586547 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 290/593] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:44.586665 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41cf8b-7dd9-4c95-a0f5-2027d70a82fa None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:44.592336 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056be8aa-7274-4e09-9503-9071987aa3d0 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:44.693328 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:44.694892 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41cf8b-7dd9-4c95-a0f5-2027d70a82fa None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:44.704290 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056be8aa-7274-4e09-9503-9071987aa3d0 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:44.736859 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed45eefe-d773-49ce-9498-62f0cb5cefee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:44.737675 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45eefe-d773-49ce-9498-62f0cb5cefee None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:44.737953 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45eefe-d773-49ce-9498-62f0cb5cefee None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:44.738227 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45eefe-d773-49ce-9498-62f0cb5cefee None admin] PATH_INFO: `/v3/projects/service` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:44.739636 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59af90-1d47-4525-848c-1f048e1b0a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:44.740304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59af90-1d47-4525-848c-1f048e1b0a83 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:44.740588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59af90-1d47-4525-848c-1f048e1b0a83 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:44.740866 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59af90-1d47-4525-848c-1f048e1b0a83 None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:44.741620 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff59af90-1d47-4525-848c-1f048e1b0a83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:44.741926 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff59af90-1d47-4525-848c-1f048e1b0a83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:44.743329 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff59af90-1d47-4525-848c-1f048e1b0a83 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:44.745422 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed45eefe-d773-49ce-9498-62f0cb5cefee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:44.746853 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed45eefe-d773-49ce-9498-62f0cb5cefee None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:44.760398 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 291/594] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:34:44.760398 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59af90-1d47-4525-848c-1f048e1b0a83 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:44.761277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-ed45eefe-d773-49ce-9498-62f0cb5cefee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:34:44.761277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:44.761277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:44.761277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:44.761277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:44.761277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:44.761277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:44.761277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:44.761277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:44.761277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:44.761277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:44.761277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:44.761277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:34:44.761277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:34:44.761277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:34:44.762314 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:34:44.762314 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:44.762314 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:44.762314 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:44.762314 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:44.762314 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:44.762314 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:44.762314 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:44.762314 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:44.762314 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:44.762314 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:44.762314 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:44.762314 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:44.762314 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:44.762314 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:44.762314 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:44.762314 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:44.763453 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:34:44.763453 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:34:44.763453 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:34:44.763453 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:34:44.763453 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:34:44.763453 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:34:44.763453 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:34:44.763453 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:44.763453 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 304/595] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:34:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:44.764050 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715b6ba-6c29-47f6-88e5-f4aff9e4a2b2 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:44.764050 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715b6ba-6c29-47f6-88e5-f4aff9e4a2b2 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:44.764360 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715b6ba-6c29-47f6-88e5-f4aff9e4a2b2 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:44.770077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 305/596] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:44.770077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715b6ba-6c29-47f6-88e5-f4aff9e4a2b2 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:44.800308 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:44.874982 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715b6ba-6c29-47f6-88e5-f4aff9e4a2b2 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:44.876097 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59af90-1d47-4525-848c-1f048e1b0a83 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:44.910970 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e6e19-dfcb-4eeb-8975-d4c50fcb5aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:44.911828 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e6e19-dfcb-4eeb-8975-d4c50fcb5aa7 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:44.912107 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e6e19-dfcb-4eeb-8975-d4c50fcb5aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:44.912386 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e6e19-dfcb-4eeb-8975-d4c50fcb5aa7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:44.914855 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01eaa28-cfaf-4f05-90a8-b9f3e124a28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:44.916120 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01eaa28-cfaf-4f05-90a8-b9f3e124a28a None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:44.916559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01eaa28-cfaf-4f05-90a8-b9f3e124a28a None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:44.916983 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01eaa28-cfaf-4f05-90a8-b9f3e124a28a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:44.919799 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e6e19-dfcb-4eeb-8975-d4c50fcb5aa7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:44.921262 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e6e19-dfcb-4eeb-8975-d4c50fcb5aa7 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:44.926081 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01eaa28-cfaf-4f05-90a8-b9f3e124a28a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:44.927753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01eaa28-cfaf-4f05-90a8-b9f3e124a28a None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:44.935719 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-b74e6e19-dfcb-4eeb-8975-d4c50fcb5aa7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:34:44.935719 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:44.935719 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:44.935719 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:44.935719 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:44.935719 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:44.935719 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:44.935719 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:44.935719 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:44.935719 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:44.935719 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:44.935719 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:44.935719 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:34:44.935719 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:34:44.935719 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:34:44.936788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:34:44.936788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:44.936788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:44.936788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:44.936788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:44.936788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:44.936788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:44.936788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:44.936788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:44.936788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:44.936788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:44.936788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:44.936788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:44.936788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:44.936788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:44.936788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:44.936788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:44.937701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:34:44.937701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:34:44.937701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:34:44.937701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:34:44.937701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:34:44.937701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:34:44.937701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:34:44.937701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:44.937701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 306/597] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:34:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:44.937701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e6e19-dfcb-4eeb-8975-d4c50fcb5aa7 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:44.945412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-a01eaa28-cfaf-4f05-90a8-b9f3e124a28a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 14:34:44.945412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:44.945412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:44.945412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:44.945412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:44.945412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:44.945412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:44.945412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:44.945412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:44.945412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:44.945412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:44.945412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:44.945412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:34:44.945412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:34:44.945412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:34:44.945412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:34:44.946627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:44.946627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:44.946627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:34:44.946627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:44.946627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:34:44.946627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:44.946627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:44.946627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:44.946627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:44.946627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:44.946627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:44.946627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:44.946627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:44.946627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:44.946627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:44.946627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:44.946627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:44.946627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:44.948026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:44.948026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:44.948026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:34:44.948026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:34:44.948026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:34:44.948026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:34:44.948026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:34:44.948026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:34:44.948026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 14:34:44.948026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:44.950310 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 292/598] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jul 20 14:34:44 2022] GET /identity/v3/users/neutron => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:44.951616 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd052320-94ec-483a-ada1-f67400a6754c None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:44.951907 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd052320-94ec-483a-ada1-f67400a6754c None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:44.952190 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd052320-94ec-483a-ada1-f67400a6754c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:44.969703 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-bd052320-94ec-483a-ada1-f67400a6754c None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:44.970910 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-bd052320-94ec-483a-ada1-f67400a6754c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:44.979140 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:45.038423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 293/599] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:45.039126 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd052320-94ec-483a-ada1-f67400a6754c None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:45.045195 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:45.056130 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e6e19-dfcb-4eeb-8975-d4c50fcb5aa7 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:45.094663 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d5cf8a-ce6f-4a80-95f0-32df0e896b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:45.095482 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5cf8a-ce6f-4a80-95f0-32df0e896b8b None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:45.095766 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5cf8a-ce6f-4a80-95f0-32df0e896b8b None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:45.096045 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5cf8a-ce6f-4a80-95f0-32df0e896b8b None admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:45.096894 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d5cf8a-ce6f-4a80-95f0-32df0e896b8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:45.097245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d5cf8a-ce6f-4a80-95f0-32df0e896b8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:45.098781 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d5cf8a-ce6f-4a80-95f0-32df0e896b8b None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:45.110562 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 307/600] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:34:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:45.111385 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d5cf8a-ce6f-4a80-95f0-32df0e896b8b None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:45.149128 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd052320-94ec-483a-ada1-f67400a6754c None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:45.184720 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420de0f5-b96f-40c0-91cc-c7a4a719ce89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:45.185707 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420de0f5-b96f-40c0-91cc-c7a4a719ce89 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:45.186182 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420de0f5-b96f-40c0-91cc-c7a4a719ce89 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:45.186637 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420de0f5-b96f-40c0-91cc-c7a4a719ce89 None admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:45.187704 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420de0f5-b96f-40c0-91cc-c7a4a719ce89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:45.188206 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420de0f5-b96f-40c0-91cc-c7a4a719ce89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:45.189896 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420de0f5-b96f-40c0-91cc-c7a4a719ce89 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:45.204006 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 294/601] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:34:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:45.205213 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420de0f5-b96f-40c0-91cc-c7a4a719ce89 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:45.210722 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:45.221860 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d5cf8a-ce6f-4a80-95f0-32df0e896b8b None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:45.258314 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72724e90-ef5b-485e-9833-56b135697667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:45.259113 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72724e90-ef5b-485e-9833-56b135697667 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:45.259751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72724e90-ef5b-485e-9833-56b135697667 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:45.260041 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72724e90-ef5b-485e-9833-56b135697667 None admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:45.260886 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72724e90-ef5b-485e-9833-56b135697667 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:45.261204 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72724e90-ef5b-485e-9833-56b135697667 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:45.262640 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72724e90-ef5b-485e-9833-56b135697667 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:45.280248 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 308/602] 10.222.0.15 () {56 vars in 1139 bytes} [Wed Jul 20 14:34:45 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:45.281078 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72724e90-ef5b-485e-9833-56b135697667 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:45.294271 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:45.336107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420de0f5-b96f-40c0-91cc-c7a4a719ce89 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:45.372040 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546b92d-77a0-475f-9af4-c1a900369503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:45.373172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546b92d-77a0-475f-9af4-c1a900369503 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:45.373600 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546b92d-77a0-475f-9af4-c1a900369503 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:45.374047 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546b92d-77a0-475f-9af4-c1a900369503 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:45.386284 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2546b92d-77a0-475f-9af4-c1a900369503 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:45.394172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-2546b92d-77a0-475f-9af4-c1a900369503 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:34:45.394172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:45.394172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:45.394172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:45.394172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:45.394172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:45.394172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:45.394172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:45.394172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:45.394172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:45.394172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:45.394172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:45.394172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:45.394172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:45.394172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:45.394172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:45.395172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:45.395172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:45.395172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:45.395172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:45.395172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:45.395172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:45.395172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:45.395172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:45.395172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:45.395172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:45.395172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:45.395172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:45.395172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:45.395172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:45.395172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:45.395172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:45.395999 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:45.395999 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:45.395999 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:45.395999 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:45.395999 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:45.395999 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:45.395999 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:45.395999 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:45.395999 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:45.395999 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:45.395999 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:34:45.395999 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:45.395999 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 295/603] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jul 20 14:34:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:45.396688 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546b92d-77a0-475f-9af4-c1a900369503 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:45.404513 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72724e90-ef5b-485e-9833-56b135697667 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:45.441793 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679bbe90-87ef-4b84-8e6e-0beeca91e845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:45.443123 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bbe90-87ef-4b84-8e6e-0beeca91e845 None admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:45.443427 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bbe90-87ef-4b84-8e6e-0beeca91e845 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:45.443712 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bbe90-87ef-4b84-8e6e-0beeca91e845 None admin] PATH_INFO: `/v3/projects/59333e4f593346e184877d6bd9d22026/users/15d7d59080cd4832a45dbb6c0349cf4c/roles/62e9fa78a7c44a16bb4540d3e8d7044d` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:45.460451 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679bbe90-87ef-4b84-8e6e-0beeca91e845 None admin] RBAC: Authorizing `identity:create_grant(project_id=59333e4f593346e184877d6bd9d22026, user_id=15d7d59080cd4832a45dbb6c0349cf4c, role_id=62e9fa78a7c44a16bb4540d3e8d7044d)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:45.461820 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679bbe90-87ef-4b84-8e6e-0beeca91e845 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:45.466887 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:45.477994 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 309/604] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jul 20 14:34:45 2022] PUT /identity/v3/projects/59333e4f593346e184877d6bd9d22026/users/15d7d59080cd4832a45dbb6c0349cf4c/roles/62e9fa78a7c44a16bb4540d3e8d7044d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:34:45.477994 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679bbe90-87ef-4b84-8e6e-0beeca91e845 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:45.532379 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546b92d-77a0-475f-9af4-c1a900369503 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:45.553312 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:45.583948 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd45409-71a8-4d9a-962b-d8f6caad88d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:45.585549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd45409-71a8-4d9a-962b-d8f6caad88d8 None admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:45.586006 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd45409-71a8-4d9a-962b-d8f6caad88d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:45.586466 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd45409-71a8-4d9a-962b-d8f6caad88d8 None admin] PATH_INFO: `/v3/projects/59333e4f593346e184877d6bd9d22026/users/9618ec9bbe76431e84640b0e5ca90eb1/roles/62e9fa78a7c44a16bb4540d3e8d7044d` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:45.605064 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679bbe90-87ef-4b84-8e6e-0beeca91e845 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:45.607549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd45409-71a8-4d9a-962b-d8f6caad88d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=59333e4f593346e184877d6bd9d22026, user_id=9618ec9bbe76431e84640b0e5ca90eb1, role_id=62e9fa78a7c44a16bb4540d3e8d7044d)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:45.610321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd45409-71a8-4d9a-962b-d8f6caad88d8 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:45.635094 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 296/605] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jul 20 14:34:45 2022] PUT /identity/v3/projects/59333e4f593346e184877d6bd9d22026/users/9618ec9bbe76431e84640b0e5ca90eb1/roles/62e9fa78a7c44a16bb4540d3e8d7044d => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:34:45.635094 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd45409-71a8-4d9a-962b-d8f6caad88d8 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:45.670029 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d075d01-c686-45a1-9f26-e4920f238276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:45.670928 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d075d01-c686-45a1-9f26-e4920f238276 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:45.671244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d075d01-c686-45a1-9f26-e4920f238276 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:45.671580 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d075d01-c686-45a1-9f26-e4920f238276 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:45.691433 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d075d01-c686-45a1-9f26-e4920f238276 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:45.693281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d075d01-c686-45a1-9f26-e4920f238276 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:45.720063 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-6d075d01-c686-45a1-9f26-e4920f238276 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:45.720063 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:45.720063 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:45.720063 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:45.720063 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:45.720063 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:45.720063 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:45.720063 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:45.720063 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:45.720063 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:45.720063 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:45.720063 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:45.720063 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:45.720063 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:45.720063 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:45.721054 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:45.721054 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:45.721054 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:45.721054 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:45.721054 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:45.721054 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:45.721054 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:45.721054 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:45.721054 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:45.721054 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:45.721054 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:45.721054 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:45.721054 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:45.721054 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:45.721054 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:45.721054 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:45.721054 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:45.721957 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:45.721957 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:45.721957 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:45.721957 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:45.724792 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 310/606] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:45.730818 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d075d01-c686-45a1-9f26-e4920f238276 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:45.777323 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd45409-71a8-4d9a-962b-d8f6caad88d8 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:45.794012 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:45.821594 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ef313-7dbd-4516-a4a6-b8d56fccac80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:45.822508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ef313-7dbd-4516-a4a6-b8d56fccac80 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:45.822894 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ef313-7dbd-4516-a4a6-b8d56fccac80 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:45.823299 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ef313-7dbd-4516-a4a6-b8d56fccac80 None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:45.824277 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ef313-7dbd-4516-a4a6-b8d56fccac80 None admin] RBAC: Adding query filter params (name=service) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:45.824687 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ef313-7dbd-4516-a4a6-b8d56fccac80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:45.826365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ef313-7dbd-4516-a4a6-b8d56fccac80 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:45.836167 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 297/607] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jul 20 14:34:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:45.842214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ef313-7dbd-4516-a4a6-b8d56fccac80 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:45.871213 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d075d01-c686-45a1-9f26-e4920f238276 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:45.886237 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:45.918504 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae68648f-4a28-49b9-ac6b-476829088260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:45.919462 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68648f-4a28-49b9-ac6b-476829088260 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:45.919908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68648f-4a28-49b9-ac6b-476829088260 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:45.920331 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68648f-4a28-49b9-ac6b-476829088260 None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:45.921474 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae68648f-4a28-49b9-ac6b-476829088260 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:45.921949 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae68648f-4a28-49b9-ac6b-476829088260 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:45.924704 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae68648f-4a28-49b9-ac6b-476829088260 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:45.944057 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 311/608] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:45.949293 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae68648f-4a28-49b9-ac6b-476829088260 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:45.964148 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ef313-7dbd-4516-a4a6-b8d56fccac80 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:46.007164 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759cbe7-0229-46d1-95b0-ef647a574d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:46.008321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759cbe7-0229-46d1-95b0-ef647a574d11 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:46.008538 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759cbe7-0229-46d1-95b0-ef647a574d11 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:46.008919 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759cbe7-0229-46d1-95b0-ef647a574d11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:46.026853 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8759cbe7-0229-46d1-95b0-ef647a574d11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:46.028521 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8759cbe7-0229-46d1-95b0-ef647a574d11 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:46.064476 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-8759cbe7-0229-46d1-95b0-ef647a574d11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:46.064476 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:46.064476 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:46.064476 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:46.064476 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:46.064476 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:46.064476 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:46.064476 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:46.064476 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:46.064476 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:46.064476 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:46.064476 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:46.064476 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:46.064476 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:46.064476 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:46.065711 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:46.065711 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:46.065711 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:46.065711 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:46.065711 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:46.065711 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:46.065711 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:46.065711 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:46.065711 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:46.065711 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:46.065711 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:46.065711 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:46.065711 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:46.065711 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:46.065711 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:46.065711 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:46.065711 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:46.066700 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:46.066700 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:46.066700 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:46.066700 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:46.069267 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 298/609] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:46.072126 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae68648f-4a28-49b9-ac6b-476829088260 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:46.075360 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759cbe7-0229-46d1-95b0-ef647a574d11 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:46.125107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:46.129058 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced9de54-5e39-4d8e-b06c-ed2925b5dd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:46.130087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced9de54-5e39-4d8e-b06c-ed2925b5dd37 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:46.130515 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced9de54-5e39-4d8e-b06c-ed2925b5dd37 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:46.130931 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced9de54-5e39-4d8e-b06c-ed2925b5dd37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:46.139533 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced9de54-5e39-4d8e-b06c-ed2925b5dd37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:46.141340 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced9de54-5e39-4d8e-b06c-ed2925b5dd37 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:46.156257 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-ced9de54-5e39-4d8e-b06c-ed2925b5dd37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:34:46.156257 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:46.156257 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:46.156257 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:46.156257 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:46.156257 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:46.156257 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:46.156257 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:46.156257 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:46.156257 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:46.156257 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:46.156257 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:46.156257 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:34:46.156257 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:34:46.156257 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:34:46.157216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:34:46.157216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:46.157216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:46.157216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:46.157216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:46.157216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:46.157216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:46.157216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:46.157216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:46.157216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:46.157216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:46.157216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:46.157216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:46.157216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:46.157216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:46.157216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:46.157216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:46.158137 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:34:46.158137 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:34:46.158137 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:34:46.158137 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:34:46.158137 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:34:46.158137 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:34:46.158137 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:34:46.158137 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:46.160900 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 312/610] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:34:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:34:46.167087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced9de54-5e39-4d8e-b06c-ed2925b5dd37 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:46.209193 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:46.242808 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759cbe7-0229-46d1-95b0-ef647a574d11 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:46.283399 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced9de54-5e39-4d8e-b06c-ed2925b5dd37 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:46.313631 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f1bebb-f107-42e4-a26f-61f4afcabd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:46.314322 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1bebb-f107-42e4-a26f-61f4afcabd5e None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:46.314607 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1bebb-f107-42e4-a26f-61f4afcabd5e None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:46.314896 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1bebb-f107-42e4-a26f-61f4afcabd5e None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:46.315675 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f1bebb-f107-42e4-a26f-61f4afcabd5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:46.315968 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f1bebb-f107-42e4-a26f-61f4afcabd5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:46.317347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f1bebb-f107-42e4-a26f-61f4afcabd5e None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:46.327465 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589cf2f0-1e0f-47fd-b59c-7b79ccf026d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:46.328370 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589cf2f0-1e0f-47fd-b59c-7b79ccf026d2 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:46.328796 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589cf2f0-1e0f-47fd-b59c-7b79ccf026d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:46.329230 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589cf2f0-1e0f-47fd-b59c-7b79ccf026d2 None admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:46.330167 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589cf2f0-1e0f-47fd-b59c-7b79ccf026d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:46.330613 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589cf2f0-1e0f-47fd-b59c-7b79ccf026d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:46.330723 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 299/611] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:34:46.332403 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589cf2f0-1e0f-47fd-b59c-7b79ccf026d2 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:46.340078 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f1bebb-f107-42e4-a26f-61f4afcabd5e None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:46.345654 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 313/612] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:34:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:34:46.353605 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ebe9e-7ffa-47ea-881c-c09b91e5090d None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:46.353605 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ebe9e-7ffa-47ea-881c-c09b91e5090d None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:46.353605 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ebe9e-7ffa-47ea-881c-c09b91e5090d None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:46.354162 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 314/613] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:46.355272 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3ebe9e-7ffa-47ea-881c-c09b91e5090d None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:46.447518 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:46.459360 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f1bebb-f107-42e4-a26f-61f4afcabd5e None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:46.472441 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:46.486891 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3ebe9e-7ffa-47ea-881c-c09b91e5090d None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:46.508732 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a5410-a934-4f60-8b81-6a51cf3d61f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:46.510010 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a5410-a934-4f60-8b81-6a51cf3d61f6 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:46.510291 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a5410-a934-4f60-8b81-6a51cf3d61f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:46.510565 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a5410-a934-4f60-8b81-6a51cf3d61f6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:46.520217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a5410-a934-4f60-8b81-6a51cf3d61f6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:46.522596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a5410-a934-4f60-8b81-6a51cf3d61f6 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:46.529537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24baa15b-b4b1-465f-94ca-c99430a448b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:46.530700 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24baa15b-b4b1-465f-94ca-c99430a448b5 None admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:46.531475 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24baa15b-b4b1-465f-94ca-c99430a448b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:46.531953 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24baa15b-b4b1-465f-94ca-c99430a448b5 None admin] PATH_INFO: `/v3/projects/59333e4f593346e184877d6bd9d22026/users/7658ffde740d446682c1876662815987/roles/62e9fa78a7c44a16bb4540d3e8d7044d` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:46.541665 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-ee8a5410-a934-4f60-8b81-6a51cf3d61f6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 14:34:46.541665 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:46.541665 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:46.541665 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:46.541665 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:46.541665 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:46.541665 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:46.541665 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:46.541665 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:46.541665 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:46.541665 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:46.541665 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:46.541665 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:34:46.541665 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:34:46.541665 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:34:46.541665 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:34:46.542596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:46.542596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:46.542596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:34:46.542596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:46.542596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:34:46.542596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:46.542596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:46.542596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:46.542596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:46.542596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:46.542596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:46.542596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:46.542596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:46.542596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:46.542596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:46.542596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:46.542596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:46.542596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:46.544132 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:46.544132 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:46.544132 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:34:46.544132 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:34:46.544132 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:34:46.544132 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:34:46.544132 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:34:46.544132 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:34:46.544132 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 14:34:46.544132 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:46.544132 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 300/614] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:34:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:46.544742 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a24b9f6-961a-40a0-b8b2-820c5a09e675 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:46.544809 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a24b9f6-961a-40a0-b8b2-820c5a09e675 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:46.545095 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a24b9f6-961a-40a0-b8b2-820c5a09e675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:46.552049 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24baa15b-b4b1-465f-94ca-c99430a448b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=59333e4f593346e184877d6bd9d22026, user_id=7658ffde740d446682c1876662815987, role_id=62e9fa78a7c44a16bb4540d3e8d7044d)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:46.554675 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24baa15b-b4b1-465f-94ca-c99430a448b5 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:46.564904 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-2a24b9f6-961a-40a0-b8b2-820c5a09e675 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:46.566198 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-2a24b9f6-961a-40a0-b8b2-820c5a09e675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:46.572637 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 315/615] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jul 20 14:34:46 2022] PUT /identity/v3/projects/59333e4f593346e184877d6bd9d22026/users/7658ffde740d446682c1876662815987/roles/62e9fa78a7c44a16bb4540d3e8d7044d => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 14:34:46.573820 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24baa15b-b4b1-465f-94ca-c99430a448b5 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:46.739172 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24baa15b-b4b1-465f-94ca-c99430a448b5 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:46.742153 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 301/616] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:46.769345 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a24b9f6-961a-40a0-b8b2-820c5a09e675 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:46.789048 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c8958-6e6f-4010-beba-8b1ed79f9600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:46.789740 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c8958-6e6f-4010-beba-8b1ed79f9600 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:46.790031 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c8958-6e6f-4010-beba-8b1ed79f9600 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:46.790291 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c8958-6e6f-4010-beba-8b1ed79f9600 None admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:46.791843 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c8958-6e6f-4010-beba-8b1ed79f9600 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:46.791843 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c8958-6e6f-4010-beba-8b1ed79f9600 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:46.792936 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c8958-6e6f-4010-beba-8b1ed79f9600 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:46.812410 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 316/617] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Jul 20 14:34:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:46.815450 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:46.818765 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c8958-6e6f-4010-beba-8b1ed79f9600 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:46.861013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:46.896744 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a24b9f6-961a-40a0-b8b2-820c5a09e675 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:46.933997 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201c328-d187-4375-ab50-ea7750c717e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:46.935751 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201c328-d187-4375-ab50-ea7750c717e2 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:46.936106 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201c328-d187-4375-ab50-ea7750c717e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:46.936449 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201c328-d187-4375-ab50-ea7750c717e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:46.950405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201c328-d187-4375-ab50-ea7750c717e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:46.959127 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-2201c328-d187-4375-ab50-ea7750c717e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:34:46.959127 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:46.959127 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:46.959127 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:46.959127 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:46.959127 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:46.959127 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:46.959127 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:46.959127 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:46.959127 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:46.959127 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:46.959127 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:46.959127 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:46.959127 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:46.959127 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:46.959127 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:46.960331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:46.960331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:46.960331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:46.960331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:46.960331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:46.960331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:46.960331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:46.960331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:46.960331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:46.960331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:46.960331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:46.960331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:46.960331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:46.960331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:46.960331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:46.960331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:46.961352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:46.961352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:46.961352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:46.961352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:46.961352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:46.961352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:46.961352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:46.961352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:46.961352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:46.961352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:46.961352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:34:46.961352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:46.961352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 302/618] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jul 20 14:34:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:34:46.966967 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201c328-d187-4375-ab50-ea7750c717e2 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:47.005520 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c8958-6e6f-4010-beba-8b1ed79f9600 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:47.076646 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3ef062-0668-4585-aab6-b59a87acf6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:47.077726 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ef062-0668-4585-aab6-b59a87acf6ee None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:47.078158 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ef062-0668-4585-aab6-b59a87acf6ee None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:47.078605 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ef062-0668-4585-aab6-b59a87acf6ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:47.095910 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3ef062-0668-4585-aab6-b59a87acf6ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:47.100273 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3ef062-0668-4585-aab6-b59a87acf6ee None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:47.127518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-cb3ef062-0668-4585-aab6-b59a87acf6ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:47.127518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:47.127518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:47.127518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:47.127518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:47.127518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:47.127518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:47.127518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:47.127518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:47.127518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:47.127518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:47.127518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:47.127518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:47.127518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:47.127518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:47.128682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:47.128682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:47.128682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:47.128682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:47.128682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:47.128682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:47.128682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:47.128682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:47.128682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:47.128682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:47.128682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:47.128682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:47.128682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:47.128682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:47.128682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:47.128682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:47.128682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:47.130051 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:47.130051 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:47.130051 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:47.130051 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:47.130051 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 317/619] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:47.135418 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201c328-d187-4375-ab50-ea7750c717e2 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:47.137925 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3ef062-0668-4585-aab6-b59a87acf6ee None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:47.182654 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588b593-a69d-42a9-bc3b-19ddf9bd3a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:47.183374 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588b593-a69d-42a9-bc3b-19ddf9bd3a39 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:47.183660 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588b593-a69d-42a9-bc3b-19ddf9bd3a39 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:47.183932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588b593-a69d-42a9-bc3b-19ddf9bd3a39 None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:47.184699 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b588b593-a69d-42a9-bc3b-19ddf9bd3a39 None admin] RBAC: Adding query filter params (name=service) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:47.185008 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b588b593-a69d-42a9-bc3b-19ddf9bd3a39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:47.186405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b588b593-a69d-42a9-bc3b-19ddf9bd3a39 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:47.195300 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:47.197006 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 303/620] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jul 20 14:34:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:47.200982 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588b593-a69d-42a9-bc3b-19ddf9bd3a39 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:47.236611 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:47.324585 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3ef062-0668-4585-aab6-b59a87acf6ee None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:47.365436 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da4057-3368-45f4-bc16-50609b0a11bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:47.366087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da4057-3368-45f4-bc16-50609b0a11bb None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:47.366378 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da4057-3368-45f4-bc16-50609b0a11bb None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:47.366654 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da4057-3368-45f4-bc16-50609b0a11bb None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:47.367415 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da4057-3368-45f4-bc16-50609b0a11bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:47.367726 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da4057-3368-45f4-bc16-50609b0a11bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:47.369328 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da4057-3368-45f4-bc16-50609b0a11bb None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:47.387315 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 318/621] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:47.396139 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da4057-3368-45f4-bc16-50609b0a11bb None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:47.411199 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588b593-a69d-42a9-bc3b-19ddf9bd3a39 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:47.479704 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f544d-a547-426b-977d-ce23743fc369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:47.480540 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f544d-a547-426b-977d-ce23743fc369 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:47.480826 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f544d-a547-426b-977d-ce23743fc369 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:47.481107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f544d-a547-426b-977d-ce23743fc369 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:47.506390 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f544d-a547-426b-977d-ce23743fc369 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:47.507967 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f544d-a547-426b-977d-ce23743fc369 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:47.546557 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-3e3f544d-a547-426b-977d-ce23743fc369 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:47.546557 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:47.546557 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:47.546557 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:47.546557 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:47.546557 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:47.546557 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:47.546557 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:47.546557 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:47.546557 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:47.546557 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:47.546557 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:47.546557 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:47.546557 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:47.546557 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:47.547573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:47.547573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:47.547573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:47.547573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:47.547573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:47.547573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:47.547573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:47.547573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:47.547573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:47.547573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:47.547573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:47.547573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:47.547573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:47.547573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:47.547573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:47.547573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:47.547573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:47.548847 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:47.548847 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:47.548847 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:47.548847 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:47.550717 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 304/622] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:47.555459 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f544d-a547-426b-977d-ce23743fc369 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:47.591020 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da4057-3368-45f4-bc16-50609b0a11bb None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:47.616762 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:47.652756 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:47.659430 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978991f-ff33-444a-8e7b-750d0b698ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:47.660624 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978991f-ff33-444a-8e7b-750d0b698ac4 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:47.661101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978991f-ff33-444a-8e7b-750d0b698ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:47.661565 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978991f-ff33-444a-8e7b-750d0b698ac4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:47.671381 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e978991f-ff33-444a-8e7b-750d0b698ac4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:47.673813 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e978991f-ff33-444a-8e7b-750d0b698ac4 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:47.691298 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-e978991f-ff33-444a-8e7b-750d0b698ac4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:34:47.691298 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:47.691298 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:47.691298 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:47.691298 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:47.691298 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:47.691298 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:47.691298 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:47.691298 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:47.691298 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:47.691298 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:47.691298 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:47.691298 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:34:47.691298 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:34:47.691298 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:34:47.692236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:34:47.692236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:47.692236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:47.692236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:47.692236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:47.692236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:47.692236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:47.692236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:47.692236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:47.692236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:47.692236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:47.692236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:47.692236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:47.692236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:47.692236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:47.692236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:47.692236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:47.693132 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:34:47.693132 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:34:47.693132 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:34:47.693132 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:34:47.693132 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:34:47.693132 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:34:47.693132 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:34:47.693132 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:47.695777 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 319/623] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:34:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:47.705441 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978991f-ff33-444a-8e7b-750d0b698ac4 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:47.761053 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f544d-a547-426b-977d-ce23743fc369 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:47.847781 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc76ec-b351-4025-b4ee-f1eb283fc05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:47.848471 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc76ec-b351-4025-b4ee-f1eb283fc05b None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:47.848758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc76ec-b351-4025-b4ee-f1eb283fc05b None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:47.849037 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc76ec-b351-4025-b4ee-f1eb283fc05b None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:47.849777 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbc76ec-b351-4025-b4ee-f1eb283fc05b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:47.850088 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbc76ec-b351-4025-b4ee-f1eb283fc05b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:47.851764 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbc76ec-b351-4025-b4ee-f1eb283fc05b None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:47.856162 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978991f-ff33-444a-8e7b-750d0b698ac4 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:47.877748 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 305/624] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:34:47.883348 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc76ec-b351-4025-b4ee-f1eb283fc05b None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:47.902508 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf1666-a302-45c4-8ca7-67eaa151259b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:47.903242 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf1666-a302-45c4-8ca7-67eaa151259b None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:47.903528 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf1666-a302-45c4-8ca7-67eaa151259b None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:47.903811 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf1666-a302-45c4-8ca7-67eaa151259b None admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:47.904628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadf1666-a302-45c4-8ca7-67eaa151259b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:47.904943 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadf1666-a302-45c4-8ca7-67eaa151259b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:47.906407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadf1666-a302-45c4-8ca7-67eaa151259b None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:47.923892 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 320/625] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:34:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:47.926452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf1666-a302-45c4-8ca7-67eaa151259b None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:47.958806 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:48.026060 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc76ec-b351-4025-b4ee-f1eb283fc05b None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:48.039502 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:48.068734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb5070c-f5e4-4d59-99ed-5cfb23c5673a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:48.070088 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5070c-f5e4-4d59-99ed-5cfb23c5673a None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:48.070509 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5070c-f5e4-4d59-99ed-5cfb23c5673a None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:48.071207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5070c-f5e4-4d59-99ed-5cfb23c5673a None admin] PATH_INFO: `/v3/users/nova` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:48.080862 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb5070c-f5e4-4d59-99ed-5cfb23c5673a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:48.082730 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb5070c-f5e4-4d59-99ed-5cfb23c5673a None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:48.096396 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf1666-a302-45c4-8ca7-67eaa151259b None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:48.099599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-8fb5070c-f5e4-4d59-99ed-5cfb23c5673a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 14:34:48.099599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:48.099599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:48.099599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:48.099599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:48.099599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:48.099599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:48.099599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:48.099599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:48.099599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:48.099599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:48.099599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:48.099599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:34:48.099599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:34:48.099599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:34:48.099599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:34:48.100627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:48.100627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:48.100627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:34:48.100627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:48.100627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:34:48.100627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:48.100627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:48.100627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:48.100627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:48.100627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:48.100627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:48.100627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:48.100627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:48.100627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:48.100627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:48.100627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:48.100627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:48.100627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:48.101622 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:48.101622 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:48.101622 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:34:48.101622 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:34:48.101622 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:34:48.101622 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:34:48.101622 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:34:48.101622 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:34:48.101622 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 14:34:48.101622 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:48.103209 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 306/626] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Jul 20 14:34:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:34:48.105255 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835eeb92-cd2b-452d-bd9f-3da373e51b85 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:48.106322 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835eeb92-cd2b-452d-bd9f-3da373e51b85 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:48.106711 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835eeb92-cd2b-452d-bd9f-3da373e51b85 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:48.108885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 307/627] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:48.109885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b55922-07d8-46d1-876b-5c3107eb762c None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:48.110244 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b55922-07d8-46d1-876b-5c3107eb762c None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:48.110615 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b55922-07d8-46d1-876b-5c3107eb762c None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:48.116378 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 308/628] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:48.116378 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b55922-07d8-46d1-876b-5c3107eb762c None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:48.139476 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50533c28-97bd-4b33-9fe1-487f2e3e54f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:48.140470 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50533c28-97bd-4b33-9fe1-487f2e3e54f2 None admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:48.140791 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50533c28-97bd-4b33-9fe1-487f2e3e54f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:48.141159 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50533c28-97bd-4b33-9fe1-487f2e3e54f2 None admin] PATH_INFO: `/v3/projects/59333e4f593346e184877d6bd9d22026/users/37f4844988474d7a91133d77906611a5/roles/62e9fa78a7c44a16bb4540d3e8d7044d` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:48.157586 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50533c28-97bd-4b33-9fe1-487f2e3e54f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=59333e4f593346e184877d6bd9d22026, user_id=37f4844988474d7a91133d77906611a5, role_id=62e9fa78a7c44a16bb4540d3e8d7044d)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:48.159319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50533c28-97bd-4b33-9fe1-487f2e3e54f2 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:48.175344 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 321/629] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jul 20 14:34:47 2022] PUT /identity/v3/projects/59333e4f593346e184877d6bd9d22026/users/37f4844988474d7a91133d77906611a5/roles/62e9fa78a7c44a16bb4540d3e8d7044d => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:34:48.176903 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e4919-bc54-4f43-985f-02a2ffef917d None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:48.177255 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e4919-bc54-4f43-985f-02a2ffef917d None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:48.177650 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e4919-bc54-4f43-985f-02a2ffef917d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:48.196802 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-e89e4919-bc54-4f43-985f-02a2ffef917d None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:48.198321 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e89e4919-bc54-4f43-985f-02a2ffef917d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:48.240971 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b55922-07d8-46d1-876b-5c3107eb762c None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:48.281675 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 322/630] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:48.283401 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7d8ff-eb72-42e2-9ea5-d812d9f801fd None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:48.283759 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7d8ff-eb72-42e2-9ea5-d812d9f801fd None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:48.284107 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7d8ff-eb72-42e2-9ea5-d812d9f801fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:48.288844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbc527-8d37-4daf-bb05-6a72037990e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:48.290217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbc527-8d37-4daf-bb05-6a72037990e2 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:48.291430 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbc527-8d37-4daf-bb05-6a72037990e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:48.291745 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbc527-8d37-4daf-bb05-6a72037990e2 None admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:48.292643 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dbc527-8d37-4daf-bb05-6a72037990e2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:48.292957 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dbc527-8d37-4daf-bb05-6a72037990e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:48.296524 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dbc527-8d37-4daf-bb05-6a72037990e2 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:48.304905 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-e7e7d8ff-eb72-42e2-9ea5-d812d9f801fd None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:48.306261 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e7e7d8ff-eb72-42e2-9ea5-d812d9f801fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:48.310612 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:48.324293 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 309/631] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Jul 20 14:34:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:48.330175 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbc527-8d37-4daf-bb05-6a72037990e2 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:48.362852 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:48.407303 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 323/632] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:48.408181 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7d8ff-eb72-42e2-9ea5-d812d9f801fd None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:48.452231 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbc527-8d37-4daf-bb05-6a72037990e2 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:48.492662 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31029da0-5a4f-4b5f-ad92-a1afa96fabd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:48.493784 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31029da0-5a4f-4b5f-ad92-a1afa96fabd7 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:48.494306 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31029da0-5a4f-4b5f-ad92-a1afa96fabd7 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:48.494830 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31029da0-5a4f-4b5f-ad92-a1afa96fabd7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:48.506595 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31029da0-5a4f-4b5f-ad92-a1afa96fabd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:48.515253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-31029da0-5a4f-4b5f-ad92-a1afa96fabd7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:34:48.515253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:48.515253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:48.515253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:48.515253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:48.515253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:48.515253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:48.515253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:48.515253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:48.515253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:48.515253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:48.515253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:48.515253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:48.515253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:48.515253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:48.515253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:48.516272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:48.516272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:48.516272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:48.516272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:48.516272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:48.516272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:48.516272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:48.516272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:48.516272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:48.516272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:48.516272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:48.516272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:48.516272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:48.516272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:48.516272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:48.516272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:48.517135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:48.517135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:48.517135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:48.517135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:48.517135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:48.517135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:48.517135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:48.517135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:48.517135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:48.517135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:48.517135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:34:48.517135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:48.520192 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 310/633] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jul 20 14:34:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:48.521240 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31029da0-5a4f-4b5f-ad92-a1afa96fabd7 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:48.549536 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7d8ff-eb72-42e2-9ea5-d812d9f801fd None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:48.594743 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656702cc-ca84-4a60-8a2e-599f111341c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:48.595462 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656702cc-ca84-4a60-8a2e-599f111341c4 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:48.595746 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656702cc-ca84-4a60-8a2e-599f111341c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:48.596030 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656702cc-ca84-4a60-8a2e-599f111341c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:48.609743 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:48.624072 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:48.639279 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31029da0-5a4f-4b5f-ad92-a1afa96fabd7 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:48.681524 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f06678-6e86-4f27-baca-4bb85c9d7487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:48.682175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656702cc-ca84-4a60-8a2e-599f111341c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:48.682996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f06678-6e86-4f27-baca-4bb85c9d7487 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:48.683674 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f06678-6e86-4f27-baca-4bb85c9d7487 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:48.683848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656702cc-ca84-4a60-8a2e-599f111341c4 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:48.684333 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f06678-6e86-4f27-baca-4bb85c9d7487 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:48.705206 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f06678-6e86-4f27-baca-4bb85c9d7487 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:48.715861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-18f06678-6e86-4f27-baca-4bb85c9d7487 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:34:48.715861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:48.715861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:48.715861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:48.715861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:48.715861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:48.715861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:48.715861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:48.715861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:48.715861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:48.715861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:48.715861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:48.715861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:48.715861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:48.715861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:48.715861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:48.716886 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:48.716886 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:48.716886 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:48.716886 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:48.716886 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:48.716886 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:48.716886 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:48.716886 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:48.716886 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:48.716886 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:48.716886 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:48.716886 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:48.716886 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:48.716886 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:48.716886 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:48.716886 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:48.717758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:48.717758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:48.717758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:48.717758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:48.717758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:48.717758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:48.717758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:48.717758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:48.717758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:48.717758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:48.717758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:34:48.717758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:48.717758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 311/634] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jul 20 14:34:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:48.718480 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f06678-6e86-4f27-baca-4bb85c9d7487 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:48.807319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 324/635] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:34:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:34:48.808835 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656702cc-ca84-4a60-8a2e-599f111341c4 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:48.844364 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f06678-6e86-4f27-baca-4bb85c9d7487 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:48.864421 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:48.882381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166f470-a4e7-436a-92ed-f3dd5562d509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:48.883799 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166f470-a4e7-436a-92ed-f3dd5562d509 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:48.884379 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166f470-a4e7-436a-92ed-f3dd5562d509 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:48.884867 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166f470-a4e7-436a-92ed-f3dd5562d509 None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:48.885895 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c166f470-a4e7-436a-92ed-f3dd5562d509 None admin] RBAC: Adding query filter params (name=service) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:48.886492 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c166f470-a4e7-436a-92ed-f3dd5562d509 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:48.888208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c166f470-a4e7-436a-92ed-f3dd5562d509 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:48.897845 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 312/636] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jul 20 14:34:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:48.899067 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166f470-a4e7-436a-92ed-f3dd5562d509 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:48.915861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:48.941577 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656702cc-ca84-4a60-8a2e-599f111341c4 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:48.978544 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b2372-5ea9-4b9e-8ab7-ae79e2e1454f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:48.979266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b2372-5ea9-4b9e-8ab7-ae79e2e1454f None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:48.979553 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b2372-5ea9-4b9e-8ab7-ae79e2e1454f None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:48.979831 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b2372-5ea9-4b9e-8ab7-ae79e2e1454f None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:48.980598 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b2372-5ea9-4b9e-8ab7-ae79e2e1454f None admin] RBAC: Adding query filter params (name=service) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:48.980909 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b2372-5ea9-4b9e-8ab7-ae79e2e1454f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:48.982301 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b2372-5ea9-4b9e-8ab7-ae79e2e1454f None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:48.999065 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 325/637] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jul 20 14:34:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:48.999065 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b2372-5ea9-4b9e-8ab7-ae79e2e1454f None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:49.018084 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166f470-a4e7-436a-92ed-f3dd5562d509 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:49.063983 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5bdc2-84ef-4a22-a7a0-7850fae9b7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:49.065024 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5bdc2-84ef-4a22-a7a0-7850fae9b7d9 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:49.065138 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5bdc2-84ef-4a22-a7a0-7850fae9b7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:49.065323 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5bdc2-84ef-4a22-a7a0-7850fae9b7d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:49.077036 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c5bdc2-84ef-4a22-a7a0-7850fae9b7d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:49.078605 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c5bdc2-84ef-4a22-a7a0-7850fae9b7d9 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:49.096134 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-a3c5bdc2-84ef-4a22-a7a0-7850fae9b7d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:49.096134 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:49.096134 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:49.096134 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:49.096134 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:49.096134 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:49.096134 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:49.096134 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:49.096134 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:49.096134 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:49.096134 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:49.096134 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:49.096134 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:49.096134 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:49.096134 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:49.097111 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:49.097111 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:49.097111 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:49.097111 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:49.097111 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:49.097111 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:49.097111 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:49.097111 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:49.097111 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:49.097111 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:49.097111 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:49.097111 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:49.097111 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:49.097111 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:49.097111 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:49.097111 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:49.097111 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:49.098031 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:49.098031 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:49.098031 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:49.098031 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:49.098031 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 313/638] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:49.099200 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e2fcf-43fd-4d1c-81a4-c5e35027b2f9 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:49.099200 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e2fcf-43fd-4d1c-81a4-c5e35027b2f9 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:49.099200 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e2fcf-43fd-4d1c-81a4-c5e35027b2f9 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:49.100561 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 314/639] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:49.103168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e2fcf-43fd-4d1c-81a4-c5e35027b2f9 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:49.136252 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b2372-5ea9-4b9e-8ab7-ae79e2e1454f None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:49.138745 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:49.170460 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:49.174196 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ad20c-7656-451b-8c2a-c69d98872585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:49.175010 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ad20c-7656-451b-8c2a-c69d98872585 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:49.175322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ad20c-7656-451b-8c2a-c69d98872585 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:49.175701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ad20c-7656-451b-8c2a-c69d98872585 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:49.187469 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ad20c-7656-451b-8c2a-c69d98872585 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:49.188957 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ad20c-7656-451b-8c2a-c69d98872585 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:49.206573 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-4d4ad20c-7656-451b-8c2a-c69d98872585 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:49.206573 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:49.206573 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:49.206573 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:49.206573 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:49.206573 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:49.206573 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:49.206573 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:49.206573 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:49.206573 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:49.206573 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:49.206573 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:49.206573 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:49.206573 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:49.206573 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:49.207724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:49.207724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:49.207724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:49.207724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:49.207724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:49.207724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:49.207724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:49.207724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:49.207724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:49.207724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:49.207724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:49.207724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:49.207724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:49.207724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:49.207724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:49.207724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:49.207724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:49.208616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:49.208616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:49.208616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:49.208616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:49.208616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 326/640] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:49.209009 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679c1cc-70a8-496a-b560-445be50ed30f None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:49.209298 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679c1cc-70a8-496a-b560-445be50ed30f None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:49.209573 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679c1cc-70a8-496a-b560-445be50ed30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:49.222615 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e2fcf-43fd-4d1c-81a4-c5e35027b2f9 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:49.226823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-0679c1cc-70a8-496a-b560-445be50ed30f None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:49.228081 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-0679c1cc-70a8-496a-b560-445be50ed30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:49.259962 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea097f4-a725-441a-b18e-3349a495bbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:49.260815 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea097f4-a725-441a-b18e-3349a495bbcd None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:49.261062 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea097f4-a725-441a-b18e-3349a495bbcd None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:49.261338 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea097f4-a725-441a-b18e-3349a495bbcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:49.272428 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea097f4-a725-441a-b18e-3349a495bbcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:49.274133 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea097f4-a725-441a-b18e-3349a495bbcd None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:49.291892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-1ea097f4-a725-441a-b18e-3349a495bbcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:49.291892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:49.291892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:49.291892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:49.291892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:49.291892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:49.291892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:49.291892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:49.291892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:49.291892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:49.291892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:49.291892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:49.291892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:49.291892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:49.291892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:49.292911 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:49.292911 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:49.292911 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:49.292911 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:49.292911 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:49.292911 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:49.292911 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:49.292911 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:49.292911 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:49.292911 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:49.292911 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:49.292911 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:49.292911 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:49.292911 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:49.292911 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:49.292911 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:49.292911 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:49.293856 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:49.293856 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:49.293856 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:49.293856 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:49.293856 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 315/641] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:49.294152 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea097f4-a725-441a-b18e-3349a495bbcd None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:49.305897 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 327/642] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:49.306750 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0679c1cc-70a8-496a-b560-445be50ed30f None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:49.405075 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea097f4-a725-441a-b18e-3349a495bbcd None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:49.410660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:49.420689 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:49.429443 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0679c1cc-70a8-496a-b560-445be50ed30f None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:49.443645 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c86b5-8b92-490d-92a9-7da0e3f2b693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:49.444423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c86b5-8b92-490d-92a9-7da0e3f2b693 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:49.444708 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c86b5-8b92-490d-92a9-7da0e3f2b693 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:49.445306 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c86b5-8b92-490d-92a9-7da0e3f2b693 None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:49.445839 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c86b5-8b92-490d-92a9-7da0e3f2b693 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:49.446153 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c86b5-8b92-490d-92a9-7da0e3f2b693 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:49.447618 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c86b5-8b92-490d-92a9-7da0e3f2b693 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:49.465467 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 316/643] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:49.465467 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c86b5-8b92-490d-92a9-7da0e3f2b693 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:49.471639 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2be1b9-c440-4daf-afbd-54a265cfcce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:49.472323 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2be1b9-c440-4daf-afbd-54a265cfcce8 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:49.472646 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2be1b9-c440-4daf-afbd-54a265cfcce8 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:49.472922 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2be1b9-c440-4daf-afbd-54a265cfcce8 None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:49.473711 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2be1b9-c440-4daf-afbd-54a265cfcce8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:49.474015 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2be1b9-c440-4daf-afbd-54a265cfcce8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:49.475570 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2be1b9-c440-4daf-afbd-54a265cfcce8 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:49.492008 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 328/644] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:49.492008 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2be1b9-c440-4daf-afbd-54a265cfcce8 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:49.582078 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c86b5-8b92-490d-92a9-7da0e3f2b693 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:49.592897 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2be1b9-c440-4daf-afbd-54a265cfcce8 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:49.618024 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49202e-d5ea-47aa-b333-92a9830ce5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:49.618725 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49202e-d5ea-47aa-b333-92a9830ce5d9 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:49.619084 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49202e-d5ea-47aa-b333-92a9830ce5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:49.619335 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49202e-d5ea-47aa-b333-92a9830ce5d9 None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:49.620102 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b49202e-d5ea-47aa-b333-92a9830ce5d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:49.620380 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b49202e-d5ea-47aa-b333-92a9830ce5d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:49.621772 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b49202e-d5ea-47aa-b333-92a9830ce5d9 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:49.628448 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f737e51e-423a-49b4-9e81-1aca58164658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:49.629260 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f737e51e-423a-49b4-9e81-1aca58164658 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:49.629545 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f737e51e-423a-49b4-9e81-1aca58164658 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:49.629822 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f737e51e-423a-49b4-9e81-1aca58164658 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:49.634788 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 317/645] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:49.635938 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49202e-d5ea-47aa-b333-92a9830ce5d9 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:49.644659 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f737e51e-423a-49b4-9e81-1aca58164658 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:49.652634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-f737e51e-423a-49b4-9e81-1aca58164658 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:34:49.652634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:49.652634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:49.652634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:49.652634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:49.652634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:49.652634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:49.652634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:49.652634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:49.652634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:49.652634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:49.652634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:49.652634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:49.652634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:49.652634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:49.652634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:49.653693 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:49.653693 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:49.653693 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:49.653693 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:49.653693 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:49.653693 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:49.653693 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:49.653693 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:49.653693 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:49.653693 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:49.653693 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:49.653693 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:49.653693 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:49.653693 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:49.653693 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:49.653693 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:49.654703 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:49.654703 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:49.654703 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:49.654703 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:49.654703 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:49.654703 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:49.654703 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:49.654703 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:49.654703 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:49.654703 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:49.654703 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:34:49.654703 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:49.658656 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 329/646] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jul 20 14:34:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:49.659518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f737e51e-423a-49b4-9e81-1aca58164658 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:49.664608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:49.677806 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:49.747886 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49202e-d5ea-47aa-b333-92a9830ce5d9 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:49.766144 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f737e51e-423a-49b4-9e81-1aca58164658 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:49.783839 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d56425-13fd-4592-9c95-ac99b3e15b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:49.784650 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d56425-13fd-4592-9c95-ac99b3e15b31 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:49.784930 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d56425-13fd-4592-9c95-ac99b3e15b31 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:49.785278 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d56425-13fd-4592-9c95-ac99b3e15b31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:49.793212 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d56425-13fd-4592-9c95-ac99b3e15b31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:49.794671 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d56425-13fd-4592-9c95-ac99b3e15b31 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:49.801006 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919261ef-5642-486e-8a9e-3f7201cb194b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:49.801820 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919261ef-5642-486e-8a9e-3f7201cb194b None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:49.802103 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919261ef-5642-486e-8a9e-3f7201cb194b None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:49.802380 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919261ef-5642-486e-8a9e-3f7201cb194b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:49.809454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-11d56425-13fd-4592-9c95-ac99b3e15b31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:34:49.809454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:49.809454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:49.809454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:49.809454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:49.809454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:49.809454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:49.809454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:49.809454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:49.809454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:49.809454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:49.809454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:49.809454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:34:49.809454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:34:49.809454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:34:49.810353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:34:49.810353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:49.810353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:49.810353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:49.810353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:49.810353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:49.810353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:49.810353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:49.810353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:49.810353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:49.810353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:49.810353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:49.810353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:49.810353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:49.810353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:49.810353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:49.810353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:49.811769 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:34:49.811769 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:34:49.811769 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:34:49.811769 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:34:49.811769 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:34:49.811769 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:34:49.811769 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:34:49.811769 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:49.811769 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 318/647] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:34:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:49.811769 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d56425-13fd-4592-9c95-ac99b3e15b31 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:49.816924 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919261ef-5642-486e-8a9e-3f7201cb194b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:49.818402 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919261ef-5642-486e-8a9e-3f7201cb194b None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:49.838175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-919261ef-5642-486e-8a9e-3f7201cb194b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 14:34:49.838175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:49.838175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:49.838175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:49.838175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:49.838175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:49.838175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:49.838175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:49.838175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:49.838175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:49.838175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:49.838175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:49.838175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:34:49.838175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:34:49.838175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:34:49.838175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:34:49.839360 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:49.839360 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:49.839360 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:34:49.839360 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:49.839360 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:34:49.839360 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:49.839360 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:49.839360 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:49.839360 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:49.839360 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:49.839360 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:49.839360 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:49.839360 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:49.839360 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:49.839360 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:49.839360 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:49.839360 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:49.839360 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:49.840318 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:49.840318 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:49.840318 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:34:49.840318 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:34:49.840318 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:34:49.840318 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:34:49.840318 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:34:49.840318 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:34:49.840318 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 14:34:49.840318 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:49.841471 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 330/648] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:34:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:49.842308 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919261ef-5642-486e-8a9e-3f7201cb194b None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:49.908300 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:49.913882 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:49.921661 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d56425-13fd-4592-9c95-ac99b3e15b31 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:49.952335 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919261ef-5642-486e-8a9e-3f7201cb194b None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:49.957845 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d53c7b-8ed5-47c3-a3d4-6c6fc564fd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:49.958878 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d53c7b-8ed5-47c3-a3d4-6c6fc564fd4d None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:49.959331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d53c7b-8ed5-47c3-a3d4-6c6fc564fd4d None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:49.959802 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d53c7b-8ed5-47c3-a3d4-6c6fc564fd4d None admin] PATH_INFO: `/v3/users/glance` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:49.967568 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d53c7b-8ed5-47c3-a3d4-6c6fc564fd4d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:49.969244 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d53c7b-8ed5-47c3-a3d4-6c6fc564fd4d None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:49.984747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-23d53c7b-8ed5-47c3-a3d4-6c6fc564fd4d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 14:34:49.984747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:49.984747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:49.984747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:49.984747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:49.984747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:49.984747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:49.984747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:49.984747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:49.984747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:49.984747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:49.984747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:49.984747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:34:49.984747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:34:49.984747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:34:49.984747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:34:49.985735 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:49.985735 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:49.985735 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:34:49.985735 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:49.985735 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:34:49.985735 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:49.985735 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:49.985735 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:49.985735 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:49.985735 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:49.985735 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:49.985735 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:49.985735 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:49.985735 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:49.985735 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:49.985735 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:49.985735 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:49.985735 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:49.986726 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:49.986726 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:49.986726 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:34:49.986726 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:34:49.986726 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:34:49.986726 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:34:49.986726 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:34:49.986726 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:34:49.986726 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 14:34:49.986726 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:49.987313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06316c54-c05d-4f4f-a210-b43d07451461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:49.987946 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06316c54-c05d-4f4f-a210-b43d07451461 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:49.988230 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06316c54-c05d-4f4f-a210-b43d07451461 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:49.988499 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06316c54-c05d-4f4f-a210-b43d07451461 None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:49.989249 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06316c54-c05d-4f4f-a210-b43d07451461 None admin] RBAC: Adding query filter params (name=service) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:49.989548 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06316c54-c05d-4f4f-a210-b43d07451461 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:49.990172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 319/649] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:34:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:49.990955 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06316c54-c05d-4f4f-a210-b43d07451461 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:49.991315 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d53c7b-8ed5-47c3-a3d4-6c6fc564fd4d None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:50.002531 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 331/650] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jul 20 14:34:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:50.003316 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06316c54-c05d-4f4f-a210-b43d07451461 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:50.102866 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d53c7b-8ed5-47c3-a3d4-6c6fc564fd4d None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:50.103968 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06316c54-c05d-4f4f-a210-b43d07451461 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:50.132028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:50.142166 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e820bdb-35f1-4886-a994-cdd232271413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:50.142337 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d52bd8-edc1-40ef-8f24-05624bd1c263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:50.142994 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d52bd8-edc1-40ef-8f24-05624bd1c263 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:50.143126 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e820bdb-35f1-4886-a994-cdd232271413 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:50.143301 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d52bd8-edc1-40ef-8f24-05624bd1c263 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:50.143576 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d52bd8-edc1-40ef-8f24-05624bd1c263 None admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:50.143676 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e820bdb-35f1-4886-a994-cdd232271413 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:50.144018 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e820bdb-35f1-4886-a994-cdd232271413 None admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:50.144421 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d52bd8-edc1-40ef-8f24-05624bd1c263 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:50.144728 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d52bd8-edc1-40ef-8f24-05624bd1c263 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:50.145058 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e820bdb-35f1-4886-a994-cdd232271413 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:50.145453 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e820bdb-35f1-4886-a994-cdd232271413 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:50.146152 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d52bd8-edc1-40ef-8f24-05624bd1c263 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:50.147069 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e820bdb-35f1-4886-a994-cdd232271413 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:50.163916 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 320/651] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:34:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:50.163916 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e820bdb-35f1-4886-a994-cdd232271413 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:50.169321 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 332/652] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Jul 20 14:34:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:50.174253 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d52bd8-edc1-40ef-8f24-05624bd1c263 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:50.180807 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:50.268376 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e820bdb-35f1-4886-a994-cdd232271413 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:50.293969 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d52bd8-edc1-40ef-8f24-05624bd1c263 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:50.301149 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647cac8-dd9a-479b-8e95-05f00cbd427c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:50.301839 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647cac8-dd9a-479b-8e95-05f00cbd427c None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:50.302114 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647cac8-dd9a-479b-8e95-05f00cbd427c None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:50.302384 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647cac8-dd9a-479b-8e95-05f00cbd427c None admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:50.303256 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f647cac8-dd9a-479b-8e95-05f00cbd427c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:50.303621 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f647cac8-dd9a-479b-8e95-05f00cbd427c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:50.305065 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f647cac8-dd9a-479b-8e95-05f00cbd427c None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:50.322629 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 321/653] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Jul 20 14:34:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:50.323464 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647cac8-dd9a-479b-8e95-05f00cbd427c None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:50.335537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39824715-6fcf-498e-a099-560161b1ae2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:50.336349 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39824715-6fcf-498e-a099-560161b1ae2d None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:50.336632 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39824715-6fcf-498e-a099-560161b1ae2d None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:50.336906 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39824715-6fcf-498e-a099-560161b1ae2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:50.347498 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39824715-6fcf-498e-a099-560161b1ae2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:50.349027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39824715-6fcf-498e-a099-560161b1ae2d None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:50.366922 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-39824715-6fcf-498e-a099-560161b1ae2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:50.366922 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:50.366922 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:50.366922 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:50.366922 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:50.366922 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:50.366922 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:50.366922 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:50.366922 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:50.366922 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:50.366922 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:50.366922 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:50.366922 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:50.366922 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:50.366922 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:50.367905 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:50.367905 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:50.367905 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:50.367905 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:50.367905 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:50.367905 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:50.367905 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:50.367905 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:50.367905 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:50.367905 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:50.367905 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:50.367905 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:50.367905 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:50.367905 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:50.367905 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:50.367905 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:50.367905 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:50.368824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:50.368824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:50.368824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:50.368824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:50.368824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 333/654] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:50.369118 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39824715-6fcf-498e-a099-560161b1ae2d None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:50.399181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:50.408389 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:50.428971 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647cac8-dd9a-479b-8e95-05f00cbd427c None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:50.463517 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64e409-b14c-4941-932f-c47a3e6e5418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:50.464178 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64e409-b14c-4941-932f-c47a3e6e5418 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:50.464448 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64e409-b14c-4941-932f-c47a3e6e5418 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:50.464752 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64e409-b14c-4941-932f-c47a3e6e5418 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:50.465579 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f64e409-b14c-4941-932f-c47a3e6e5418 None admin] RBAC: Adding query filter params (role.id=62e9fa78a7c44a16bb4540d3e8d7044d, scope.project.id=59333e4f593346e184877d6bd9d22026, user.id=5d4a107ed04348b59d20d1703dbe04eb) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:50.466125 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f64e409-b14c-4941-932f-c47a3e6e5418 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:50.467356 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f64e409-b14c-4941-932f-c47a3e6e5418 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:50.473446 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 322/655] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:34:50 2022] GET /identity/v3/role_assignments?user.id=5d4a107ed04348b59d20d1703dbe04eb&scope.project.id=59333e4f593346e184877d6bd9d22026&role.id=62e9fa78a7c44a16bb4540d3e8d7044d => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:50.474805 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73d249-636d-419c-92c3-79ba5bfa0614 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:50.475222 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73d249-636d-419c-92c3-79ba5bfa0614 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:50.475436 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73d249-636d-419c-92c3-79ba5bfa0614 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:50.479216 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 323/656] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:50.479216 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac73d249-636d-419c-92c3-79ba5bfa0614 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:50.482764 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39824715-6fcf-498e-a099-560161b1ae2d None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:50.523874 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb58b3-205b-448f-9212-067987de40f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:50.524478 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb58b3-205b-448f-9212-067987de40f8 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:50.524880 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb58b3-205b-448f-9212-067987de40f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:50.525610 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb58b3-205b-448f-9212-067987de40f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:50.602688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb58b3-205b-448f-9212-067987de40f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:50.604374 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb58b3-205b-448f-9212-067987de40f8 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:50.609801 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac73d249-636d-419c-92c3-79ba5bfa0614 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:50.643001 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:50.657069 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf52f19-6dcb-4c5b-8778-ad58497198d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:50.657764 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf52f19-6dcb-4c5b-8778-ad58497198d6 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:50.658035 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf52f19-6dcb-4c5b-8778-ad58497198d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:50.658312 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf52f19-6dcb-4c5b-8778-ad58497198d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:50.662895 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:50.716215 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 334/657] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:34:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:34:50.716991 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb58b3-205b-448f-9212-067987de40f8 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:50.741085 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf52f19-6dcb-4c5b-8778-ad58497198d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:50.743037 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf52f19-6dcb-4c5b-8778-ad58497198d6 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:50.823011 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb58b3-205b-448f-9212-067987de40f8 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:50.856376 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 324/658] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:34:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:34:50.856376 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25593ca-342e-42db-9ef3-bca1063a8cd8 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:50.856376 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25593ca-342e-42db-9ef3-bca1063a8cd8 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:50.856376 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25593ca-342e-42db-9ef3-bca1063a8cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:50.864439 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97531683-cc52-4d0f-96d8-6031a79b9b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:50.865207 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97531683-cc52-4d0f-96d8-6031a79b9b21 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:50.865494 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97531683-cc52-4d0f-96d8-6031a79b9b21 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:50.865774 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97531683-cc52-4d0f-96d8-6031a79b9b21 None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:50.866626 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97531683-cc52-4d0f-96d8-6031a79b9b21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:50.866941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97531683-cc52-4d0f-96d8-6031a79b9b21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:50.868656 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97531683-cc52-4d0f-96d8-6031a79b9b21 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:50.871063 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-e25593ca-342e-42db-9ef3-bca1063a8cd8 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:50.871775 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:50.872273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-e25593ca-342e-42db-9ef3-bca1063a8cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:50.881752 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 335/659] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:50.882409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97531683-cc52-4d0f-96d8-6031a79b9b21 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:50.910852 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:50.953601 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 325/660] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:50.954465 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25593ca-342e-42db-9ef3-bca1063a8cd8 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:51.005570 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97531683-cc52-4d0f-96d8-6031a79b9b21 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:51.045784 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63352e2-c17b-4f50-bb67-c50a53a28baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:51.046712 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63352e2-c17b-4f50-bb67-c50a53a28baa None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:51.047285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63352e2-c17b-4f50-bb67-c50a53a28baa None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:51.047403 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63352e2-c17b-4f50-bb67-c50a53a28baa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:51.059836 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63352e2-c17b-4f50-bb67-c50a53a28baa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:51.061451 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63352e2-c17b-4f50-bb67-c50a53a28baa None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:51.063150 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25593ca-342e-42db-9ef3-bca1063a8cd8 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:51.080191 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-c63352e2-c17b-4f50-bb67-c50a53a28baa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:51.080191 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:51.080191 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:51.080191 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:51.080191 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:51.080191 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:51.080191 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:51.080191 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:51.080191 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:51.080191 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:51.080191 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:51.080191 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:51.080191 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:51.080191 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:51.080191 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:51.081285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:51.081285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:51.081285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:51.081285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:51.081285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:51.081285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:51.081285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:51.081285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:51.081285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:51.081285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:51.081285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:51.081285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:51.081285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:51.081285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:51.081285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:51.081285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:51.081285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:51.082329 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:51.082329 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:51.082329 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:51.082329 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:51.082329 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 336/661] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:51.087534 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63352e2-c17b-4f50-bb67-c50a53a28baa None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:51.100744 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c4315-ed3f-4a1e-a8f2-56b3282fe75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:51.101684 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c4315-ed3f-4a1e-a8f2-56b3282fe75e None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:51.102074 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c4315-ed3f-4a1e-a8f2-56b3282fe75e None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:51.102428 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c4315-ed3f-4a1e-a8f2-56b3282fe75e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:51.112938 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c4315-ed3f-4a1e-a8f2-56b3282fe75e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:51.114420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c4315-ed3f-4a1e-a8f2-56b3282fe75e None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:51.131929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-2d9c4315-ed3f-4a1e-a8f2-56b3282fe75e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:51.131929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:51.131929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:51.131929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:51.131929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:51.131929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:51.131929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:51.131929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:51.131929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:51.131929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:51.131929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:51.131929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:51.131929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:51.131929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:51.131929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:51.133022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:51.133022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:51.133022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:51.133022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:51.133022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:51.133022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:51.133022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:51.133022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:51.133022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:51.133022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:51.133022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:51.133022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:51.133022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:51.133022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:51.133022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:51.133022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:51.133022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:51.133931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:51.133931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:51.133931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:51.133931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:51.133931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 326/662] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:51.134300 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c4315-ed3f-4a1e-a8f2-56b3282fe75e None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:51.144698 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:51.156485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:51.204911 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63352e2-c17b-4f50-bb67-c50a53a28baa None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:51.240780 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c4315-ed3f-4a1e-a8f2-56b3282fe75e None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:51.241452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc33c561-cbdb-4c09-8ba0-3bffa91ebdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:51.242449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33c561-cbdb-4c09-8ba0-3bffa91ebdf9 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:51.242840 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33c561-cbdb-4c09-8ba0-3bffa91ebdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:51.243254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33c561-cbdb-4c09-8ba0-3bffa91ebdf9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:51.251464 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc33c561-cbdb-4c09-8ba0-3bffa91ebdf9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:51.253166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc33c561-cbdb-4c09-8ba0-3bffa91ebdf9 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:51.268751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-fc33c561-cbdb-4c09-8ba0-3bffa91ebdf9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 14:34:51.268751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:51.268751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:51.268751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:51.268751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:51.268751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:51.268751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:51.268751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:51.268751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:51.268751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:51.268751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:51.268751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:51.268751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:34:51.268751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:34:51.268751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:34:51.268751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:34:51.269966 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:51.269966 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:51.269966 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:34:51.269966 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:51.269966 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:34:51.269966 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:51.269966 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:51.269966 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:51.269966 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:51.269966 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:51.269966 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:51.269966 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:51.269966 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:51.269966 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:51.269966 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:51.269966 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:51.269966 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:51.269966 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:51.271244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:51.271244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:51.271244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:34:51.271244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:34:51.271244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:34:51.271244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:34:51.271244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:34:51.271244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:34:51.271244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 14:34:51.271244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:51.271800 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 337/663] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jul 20 14:34:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:51.272423 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc33c561-cbdb-4c09-8ba0-3bffa91ebdf9 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:51.282699 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca87e3f-5828-4633-8666-60c3c300a9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:51.283543 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca87e3f-5828-4633-8666-60c3c300a9eb None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:51.283830 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca87e3f-5828-4633-8666-60c3c300a9eb None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:51.284077 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca87e3f-5828-4633-8666-60c3c300a9eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:51.295827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca87e3f-5828-4633-8666-60c3c300a9eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:51.304088 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-4ca87e3f-5828-4633-8666-60c3c300a9eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:34:51.304088 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:51.304088 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:51.304088 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:51.304088 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:51.304088 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:51.304088 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:51.304088 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:51.304088 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:51.304088 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:51.304088 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:51.304088 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:51.304088 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:51.304088 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:51.304088 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:51.304088 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:51.305026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:51.305026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:51.305026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:51.305026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:51.305026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:51.305026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:51.305026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:51.305026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:51.305026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:51.305026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:51.305026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:51.305026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:51.305026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:51.305026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:51.305026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:51.305026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:51.305907 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:51.305907 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:51.305907 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:51.305907 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:51.305907 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:51.305907 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:51.305907 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:51.305907 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:51.305907 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:51.305907 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:51.305907 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:34:51.305907 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:51.305907 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 327/664] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jul 20 14:34:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:51.306604 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca87e3f-5828-4633-8666-60c3c300a9eb None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:51.381628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc33c561-cbdb-4c09-8ba0-3bffa91ebdf9 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:51.391001 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:51.391215 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:51.414665 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca87e3f-5828-4633-8666-60c3c300a9eb None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:51.420112 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73149f1-c089-4343-8f62-104ee47c85b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:51.420823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73149f1-c089-4343-8f62-104ee47c85b6 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:51.421112 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73149f1-c089-4343-8f62-104ee47c85b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:51.421406 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73149f1-c089-4343-8f62-104ee47c85b6 None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:51.422568 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73149f1-c089-4343-8f62-104ee47c85b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:51.423222 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73149f1-c089-4343-8f62-104ee47c85b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:51.425075 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73149f1-c089-4343-8f62-104ee47c85b6 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:51.442583 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 338/665] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:51.442583 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73149f1-c089-4343-8f62-104ee47c85b6 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:51.458743 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fb8012-d683-4f97-98d6-da8d65f72e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:51.459511 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb8012-d683-4f97-98d6-da8d65f72e81 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:51.459831 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb8012-d683-4f97-98d6-da8d65f72e81 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:51.460115 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb8012-d683-4f97-98d6-da8d65f72e81 None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:51.461311 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fb8012-d683-4f97-98d6-da8d65f72e81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:51.461632 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fb8012-d683-4f97-98d6-da8d65f72e81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:51.463517 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fb8012-d683-4f97-98d6-da8d65f72e81 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:51.485466 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 328/666] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:51.490675 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fb8012-d683-4f97-98d6-da8d65f72e81 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:51.542426 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73149f1-c089-4343-8f62-104ee47c85b6 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:51.581993 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b411f-7ba0-48c1-b8cc-8b9071d6f603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:51.582692 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b411f-7ba0-48c1-b8cc-8b9071d6f603 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:51.582974 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b411f-7ba0-48c1-b8cc-8b9071d6f603 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:51.583282 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b411f-7ba0-48c1-b8cc-8b9071d6f603 None admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:51.584139 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49b411f-7ba0-48c1-b8cc-8b9071d6f603 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:51.584455 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49b411f-7ba0-48c1-b8cc-8b9071d6f603 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:51.586026 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49b411f-7ba0-48c1-b8cc-8b9071d6f603 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:51.599266 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fb8012-d683-4f97-98d6-da8d65f72e81 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:51.612676 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 339/667] 10.222.0.15 () {56 vars in 1139 bytes} [Wed Jul 20 14:34:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:51.612676 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b411f-7ba0-48c1-b8cc-8b9071d6f603 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:51.633985 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc085f-bc22-4f75-a7a6-bef6581eac0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:51.634646 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc085f-bc22-4f75-a7a6-bef6581eac0d None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:51.634934 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc085f-bc22-4f75-a7a6-bef6581eac0d None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:51.635233 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc085f-bc22-4f75-a7a6-bef6581eac0d None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:51.635992 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bc085f-bc22-4f75-a7a6-bef6581eac0d None admin] RBAC: Adding query filter params (name=service) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:51.636313 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bc085f-bc22-4f75-a7a6-bef6581eac0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:51.637732 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bc085f-bc22-4f75-a7a6-bef6581eac0d None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:51.643916 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:51.652465 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 329/668] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jul 20 14:34:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:51.652668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:51.653446 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc085f-bc22-4f75-a7a6-bef6581eac0d None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:51.733398 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b411f-7ba0-48c1-b8cc-8b9071d6f603 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:51.757539 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc085f-bc22-4f75-a7a6-bef6581eac0d None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:51.770880 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed115d-2414-4d57-a5ba-a5946e27b0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:51.771751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed115d-2414-4d57-a5ba-a5946e27b0ad None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:51.772027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed115d-2414-4d57-a5ba-a5946e27b0ad None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:51.772369 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed115d-2414-4d57-a5ba-a5946e27b0ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:51.780035 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed115d-2414-4d57-a5ba-a5946e27b0ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:51.781621 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed115d-2414-4d57-a5ba-a5946e27b0ad None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:51.790930 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a7ec11-9ded-4a24-a863-8a8c0bd7c0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:51.791751 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7ec11-9ded-4a24-a863-8a8c0bd7c0f0 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:51.792034 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7ec11-9ded-4a24-a863-8a8c0bd7c0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:51.792307 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7ec11-9ded-4a24-a863-8a8c0bd7c0f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:51.796581 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-19ed115d-2414-4d57-a5ba-a5946e27b0ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:34:51.796581 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:51.796581 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:51.796581 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:51.796581 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:51.796581 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:51.796581 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:51.796581 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:51.796581 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:51.796581 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:51.796581 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:51.796581 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:51.796581 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:34:51.796581 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:34:51.796581 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:34:51.797454 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:34:51.797454 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:51.797454 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:51.797454 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:51.797454 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:51.797454 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:51.797454 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:51.797454 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:51.797454 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:51.797454 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:51.797454 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:51.797454 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:51.797454 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:51.797454 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:51.797454 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:51.797454 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:51.797454 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:51.798673 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:34:51.798673 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:34:51.798673 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:34:51.798673 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:34:51.798673 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:34:51.798673 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:34:51.798673 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:34:51.798673 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:51.798673 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 340/669] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:34:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:51.799626 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed115d-2414-4d57-a5ba-a5946e27b0ad None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:51.804143 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a7ec11-9ded-4a24-a863-8a8c0bd7c0f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:51.805712 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a7ec11-9ded-4a24-a863-8a8c0bd7c0f0 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:51.820120 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-59a7ec11-9ded-4a24-a863-8a8c0bd7c0f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:34:51.820120 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:51.820120 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:51.820120 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:51.820120 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:51.820120 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:51.820120 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:51.820120 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:51.820120 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:51.820120 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:51.820120 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:51.820120 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:51.820120 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:34:51.820120 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:34:51.820120 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:34:51.821017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:34:51.821017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:51.821017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:51.821017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:51.821017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:51.821017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:51.821017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:51.821017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:51.821017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:51.821017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:51.821017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:51.821017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:51.821017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:51.821017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:51.821017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:51.821017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:51.821017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:51.822163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:34:51.822163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:34:51.822163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:34:51.822163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:34:51.822163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:34:51.822163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:34:51.822163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:34:51.822163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:51.822163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 330/670] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:34:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:51.822163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a7ec11-9ded-4a24-a863-8a8c0bd7c0f0 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:51.883652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:51.894128 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:51.913646 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed115d-2414-4d57-a5ba-a5946e27b0ad None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:51.929837 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a7ec11-9ded-4a24-a863-8a8c0bd7c0f0 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:51.950098 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb67a8f7-1e97-42c5-82b3-6fc4d1b00e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:51.951013 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67a8f7-1e97-42c5-82b3-6fc4d1b00e24 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:51.951393 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67a8f7-1e97-42c5-82b3-6fc4d1b00e24 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:51.951746 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67a8f7-1e97-42c5-82b3-6fc4d1b00e24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:51.963519 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76bc664-759d-499d-af5a-5208d359ffd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:51.964331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76bc664-759d-499d-af5a-5208d359ffd5 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:51.964608 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76bc664-759d-499d-af5a-5208d359ffd5 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:51.964880 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76bc664-759d-499d-af5a-5208d359ffd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:51.975303 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76bc664-759d-499d-af5a-5208d359ffd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:51.977188 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76bc664-759d-499d-af5a-5208d359ffd5 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:51.995530 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-d76bc664-759d-499d-af5a-5208d359ffd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:51.995530 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:51.995530 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:51.995530 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:51.995530 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:51.995530 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:51.995530 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:51.995530 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:51.995530 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:51.995530 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:51.995530 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:51.995530 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:51.995530 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:51.995530 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:51.995530 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:51.996578 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:51.996578 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:51.996578 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:51.996578 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:51.996578 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:51.996578 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:51.996578 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:51.996578 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:51.996578 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:51.996578 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:51.996578 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:51.996578 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:51.996578 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:51.996578 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:51.996578 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:51.996578 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:51.996578 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:51.997477 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:51.997477 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:51.997477 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:51.997477 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:51.997477 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 331/671] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:51.998206 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76bc664-759d-499d-af5a-5208d359ffd5 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:52.033286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb67a8f7-1e97-42c5-82b3-6fc4d1b00e24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:52.034872 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb67a8f7-1e97-42c5-82b3-6fc4d1b00e24 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:52.106644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76bc664-759d-499d-af5a-5208d359ffd5 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:52.120472 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:52.124154 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:52.147039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b595e2e-2523-4b4d-8e5a-08be086cbfc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:52.147688 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b595e2e-2523-4b4d-8e5a-08be086cbfc7 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:52.147974 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b595e2e-2523-4b4d-8e5a-08be086cbfc7 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:52.148254 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b595e2e-2523-4b4d-8e5a-08be086cbfc7 None admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:52.148905 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 341/672] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:34:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:34:52.149537 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b595e2e-2523-4b4d-8e5a-08be086cbfc7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:52.149972 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb67a8f7-1e97-42c5-82b3-6fc4d1b00e24 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:52.150090 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b595e2e-2523-4b4d-8e5a-08be086cbfc7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:52.151612 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b595e2e-2523-4b4d-8e5a-08be086cbfc7 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:52.169438 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 332/673] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:34:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:52.169438 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b595e2e-2523-4b4d-8e5a-08be086cbfc7 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:52.260975 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb67a8f7-1e97-42c5-82b3-6fc4d1b00e24 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:52.277905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b595e2e-2523-4b4d-8e5a-08be086cbfc7 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:52.301010 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb377536-ce10-451f-ac4b-09fee271c46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:52.301772 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb377536-ce10-451f-ac4b-09fee271c46d None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:52.301955 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb377536-ce10-451f-ac4b-09fee271c46d None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:52.302231 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb377536-ce10-451f-ac4b-09fee271c46d None admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:52.303132 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb377536-ce10-451f-ac4b-09fee271c46d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:52.303369 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb377536-ce10-451f-ac4b-09fee271c46d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:52.304812 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb377536-ce10-451f-ac4b-09fee271c46d None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:52.315508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a1fca-fe79-44fa-ab8d-f5927a75e116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:52.316191 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a1fca-fe79-44fa-ab8d-f5927a75e116 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:52.321578 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a1fca-fe79-44fa-ab8d-f5927a75e116 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:52.321737 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 342/674] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:34:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:52.321829 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a1fca-fe79-44fa-ab8d-f5927a75e116 None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:52.321829 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a1fca-fe79-44fa-ab8d-f5927a75e116 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:52.321972 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb377536-ce10-451f-ac4b-09fee271c46d None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:52.322043 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a1fca-fe79-44fa-ab8d-f5927a75e116 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:52.322043 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a1fca-fe79-44fa-ab8d-f5927a75e116 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:52.333114 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 333/675] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:52.338233 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a1fca-fe79-44fa-ab8d-f5927a75e116 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:52.369295 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:52.372351 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:52.437526 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb377536-ce10-451f-ac4b-09fee271c46d None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:52.441588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a1fca-fe79-44fa-ab8d-f5927a75e116 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:52.477845 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ad414-a6d4-49f3-9773-b74341d9dd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:52.477845 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ad414-a6d4-49f3-9773-b74341d9dd57 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:52.477845 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ad414-a6d4-49f3-9773-b74341d9dd57 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:52.477845 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ad414-a6d4-49f3-9773-b74341d9dd57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:52.484333 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc5d08f-e4e3-4765-a6c1-4b0f87147861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:52.485010 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5d08f-e4e3-4765-a6c1-4b0f87147861 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:52.485297 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5d08f-e4e3-4765-a6c1-4b0f87147861 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:52.485579 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5d08f-e4e3-4765-a6c1-4b0f87147861 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:52.486429 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc5d08f-e4e3-4765-a6c1-4b0f87147861 None admin] RBAC: Adding query filter params (role.id=62e9fa78a7c44a16bb4540d3e8d7044d, scope.project.id=59333e4f593346e184877d6bd9d22026, user.id=9618ec9bbe76431e84640b0e5ca90eb1) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:52.487228 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc5d08f-e4e3-4765-a6c1-4b0f87147861 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:52.489068 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc5d08f-e4e3-4765-a6c1-4b0f87147861 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:52.490842 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ad414-a6d4-49f3-9773-b74341d9dd57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:52.492985 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ad414-a6d4-49f3-9773-b74341d9dd57 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:52.507348 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 334/676] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:34:52 2022] GET /identity/v3/role_assignments?user.id=9618ec9bbe76431e84640b0e5ca90eb1&scope.project.id=59333e4f593346e184877d6bd9d22026&role.id=62e9fa78a7c44a16bb4540d3e8d7044d => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:52.507348 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc5d08f-e4e3-4765-a6c1-4b0f87147861 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:52.511944 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-4e7ad414-a6d4-49f3-9773-b74341d9dd57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:52.511944 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:52.511944 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:52.511944 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:52.511944 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:52.511944 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:52.511944 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:52.511944 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:52.511944 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:52.511944 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:52.511944 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:52.511944 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:52.511944 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:52.511944 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:52.511944 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:52.512915 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:52.512915 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:52.512915 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:52.512915 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:52.512915 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:52.512915 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:52.512915 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:52.512915 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:52.512915 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:52.512915 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:52.512915 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:52.512915 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:52.512915 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:52.512915 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:52.512915 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:52.512915 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:52.512915 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:52.513836 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:52.513836 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:52.513836 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:52.513836 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:52.513836 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 343/677] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:52.514606 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ad414-a6d4-49f3-9773-b74341d9dd57 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:52.621901 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:52.629192 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc5d08f-e4e3-4765-a6c1-4b0f87147861 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:52.637110 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:52.643140 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ad414-a6d4-49f3-9773-b74341d9dd57 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:52.665662 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff757e-22fe-44c8-8888-e29781f05c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:52.666469 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff757e-22fe-44c8-8888-e29781f05c94 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:52.666751 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff757e-22fe-44c8-8888-e29781f05c94 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:52.667026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff757e-22fe-44c8-8888-e29781f05c94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:52.667916 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff757e-22fe-44c8-8888-e29781f05c94 None admin] RBAC: Adding query filter params (role.id=62e9fa78a7c44a16bb4540d3e8d7044d, scope.project.id=59333e4f593346e184877d6bd9d22026, user.id=15d7d59080cd4832a45dbb6c0349cf4c) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:52.668228 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff757e-22fe-44c8-8888-e29781f05c94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:52.669702 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff757e-22fe-44c8-8888-e29781f05c94 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:52.679958 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 335/678] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:34:52 2022] GET /identity/v3/role_assignments?user.id=15d7d59080cd4832a45dbb6c0349cf4c&scope.project.id=59333e4f593346e184877d6bd9d22026&role.id=62e9fa78a7c44a16bb4540d3e8d7044d => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:52.681548 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff757e-22fe-44c8-8888-e29781f05c94 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:52.695181 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc2ec80-fcea-468d-a577-9a96798d91d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:52.696110 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2ec80-fcea-468d-a577-9a96798d91d8 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:52.696461 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2ec80-fcea-468d-a577-9a96798d91d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:52.696814 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2ec80-fcea-468d-a577-9a96798d91d8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:52.709073 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc2ec80-fcea-468d-a577-9a96798d91d8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:52.710761 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc2ec80-fcea-468d-a577-9a96798d91d8 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:52.729069 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-ffc2ec80-fcea-468d-a577-9a96798d91d8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 14:34:52.729069 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:52.729069 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:52.729069 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:52.729069 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:52.729069 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:52.729069 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:52.729069 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:52.729069 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:52.729069 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:52.729069 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:52.729069 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:52.729069 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:34:52.729069 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:34:52.729069 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:34:52.729069 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:34:52.730097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:52.730097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:52.730097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:34:52.730097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:52.730097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:34:52.730097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:52.730097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:52.730097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:52.730097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:52.730097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:52.730097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:52.730097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:52.730097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:52.730097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:52.730097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:52.730097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:52.730097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:52.730097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:52.731105 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:52.731105 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:52.731105 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:34:52.731105 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:34:52.731105 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:34:52.731105 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:34:52.731105 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:34:52.731105 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:34:52.731105 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 14:34:52.731105 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:52.733249 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 344/679] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:34:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:52.735093 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c436b5-b10f-47f2-8a05-97705dc2b344 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:52.736085 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c436b5-b10f-47f2-8a05-97705dc2b344 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:52.736364 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c436b5-b10f-47f2-8a05-97705dc2b344 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:52.737981 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 345/680] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:52.743314 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c436b5-b10f-47f2-8a05-97705dc2b344 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:52.819214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff757e-22fe-44c8-8888-e29781f05c94 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:52.870528 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd31b89-6c67-4fd5-bd2d-c6bd9d4dcd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:52.871324 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd31b89-6c67-4fd5-bd2d-c6bd9d4dcd39 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:52.871624 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd31b89-6c67-4fd5-bd2d-c6bd9d4dcd39 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:52.871900 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd31b89-6c67-4fd5-bd2d-c6bd9d4dcd39 None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:52.872833 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd31b89-6c67-4fd5-bd2d-c6bd9d4dcd39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:52.873191 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd31b89-6c67-4fd5-bd2d-c6bd9d4dcd39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:52.874654 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd31b89-6c67-4fd5-bd2d-c6bd9d4dcd39 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:52.877613 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c436b5-b10f-47f2-8a05-97705dc2b344 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:52.888888 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 336/681] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:52.891425 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11ad6d-4573-4907-9620-7bc56e0ab6fa None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:52.891675 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11ad6d-4573-4907-9620-7bc56e0ab6fa None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:52.892053 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11ad6d-4573-4907-9620-7bc56e0ab6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:52.911781 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-7f11ad6d-4573-4907-9620-7bc56e0ab6fa None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:52.913221 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-7f11ad6d-4573-4907-9620-7bc56e0ab6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:52.916859 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0959f19-3a59-471e-b139-6b280fab33ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:52.917102 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0959f19-3a59-471e-b139-6b280fab33ec None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:52.917388 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0959f19-3a59-471e-b139-6b280fab33ec None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:52.917660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0959f19-3a59-471e-b139-6b280fab33ec None admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:52.918508 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0959f19-3a59-471e-b139-6b280fab33ec None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:52.918820 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0959f19-3a59-471e-b139-6b280fab33ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:52.920301 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0959f19-3a59-471e-b139-6b280fab33ec None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:52.930823 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:52.945903 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 346/682] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Jul 20 14:34:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:52.946806 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0959f19-3a59-471e-b139-6b280fab33ec None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:52.971426 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:52.999235 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 337/683] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:53.000387 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f11ad6d-4573-4907-9620-7bc56e0ab6fa None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:53.073496 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0959f19-3a59-471e-b139-6b280fab33ec None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:53.112411 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8999f7d-0561-4d80-ab63-06e04c9f06e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:53.113416 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8999f7d-0561-4d80-ab63-06e04c9f06e6 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:53.113619 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8999f7d-0561-4d80-ab63-06e04c9f06e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:53.113884 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8999f7d-0561-4d80-ab63-06e04c9f06e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:53.121573 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8999f7d-0561-4d80-ab63-06e04c9f06e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:53.123078 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8999f7d-0561-4d80-ab63-06e04c9f06e6 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:53.128405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f11ad6d-4573-4907-9620-7bc56e0ab6fa None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:53.139071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-d8999f7d-0561-4d80-ab63-06e04c9f06e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:34:53.139071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:53.139071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:53.139071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:53.139071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:53.139071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:53.139071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:53.139071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:53.139071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:53.139071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:53.139071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:53.139071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:53.139071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:34:53.139071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:34:53.139071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:34:53.140016 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:34:53.140016 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:53.140016 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:53.140016 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:53.140016 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:53.140016 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:53.140016 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:53.140016 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:53.140016 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:53.140016 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:53.140016 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:53.140016 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:53.140016 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:53.140016 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:53.140016 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:53.140016 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:53.140016 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:53.141359 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:34:53.141359 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:34:53.141359 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:34:53.141359 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:34:53.141359 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:34:53.141359 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:34:53.141359 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:34:53.141359 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:53.141359 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 347/684] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:34:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:53.142083 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8999f7d-0561-4d80-ab63-06e04c9f06e6 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:53.180920 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05b05e-6efa-40e8-8ecc-ffd2f920486e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:53.182464 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05b05e-6efa-40e8-8ecc-ffd2f920486e None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:53.182932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05b05e-6efa-40e8-8ecc-ffd2f920486e None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:53.183385 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05b05e-6efa-40e8-8ecc-ffd2f920486e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:53.210661 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:53.246392 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:53.278657 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e05b05e-6efa-40e8-8ecc-ffd2f920486e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:53.278857 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8999f7d-0561-4d80-ab63-06e04c9f06e6 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:53.280177 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e05b05e-6efa-40e8-8ecc-ffd2f920486e None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:53.326671 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bebb6ad-5a29-431f-8c6c-610baed9200a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:53.328008 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebb6ad-5a29-431f-8c6c-610baed9200a None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:53.328709 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebb6ad-5a29-431f-8c6c-610baed9200a None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:53.329157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebb6ad-5a29-431f-8c6c-610baed9200a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:53.345497 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bebb6ad-5a29-431f-8c6c-610baed9200a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:53.357020 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-7bebb6ad-5a29-431f-8c6c-610baed9200a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:34:53.357020 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:53.357020 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:53.357020 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:53.357020 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:53.357020 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:53.357020 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:53.357020 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:53.357020 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:53.357020 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:53.357020 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:53.357020 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:53.357020 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:53.357020 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:53.357020 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:53.357020 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:53.358028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:53.358028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:53.358028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:53.358028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:53.358028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:53.358028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:53.358028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:53.358028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:53.358028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:53.358028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:53.358028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:53.358028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:53.358028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:53.358028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:53.358028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:53.358028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:53.359178 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:53.359178 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:53.359178 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:53.359178 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:53.359178 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:53.359178 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:53.359178 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:53.359178 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:53.359178 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:53.359178 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:53.359178 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:34:53.359178 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:53.359178 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 348/685] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:34:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:53.360781 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bebb6ad-5a29-431f-8c6c-610baed9200a None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:53.402730 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 338/686] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:34:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:34:53.403933 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05b05e-6efa-40e8-8ecc-ffd2f920486e None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:53.470659 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:53.474420 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bebb6ad-5a29-431f-8c6c-610baed9200a None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:53.527427 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9760c-28f7-4f4f-bf58-cf2c157e0e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:53.528653 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9760c-28f7-4f4f-bf58-cf2c157e0e83 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:53.529142 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9760c-28f7-4f4f-bf58-cf2c157e0e83 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:53.529628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9760c-28f7-4f4f-bf58-cf2c157e0e83 None admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:53.530790 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9760c-28f7-4f4f-bf58-cf2c157e0e83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:53.531386 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9760c-28f7-4f4f-bf58-cf2c157e0e83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:53.533189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9760c-28f7-4f4f-bf58-cf2c157e0e83 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:53.536637 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:53.537048 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05b05e-6efa-40e8-8ecc-ffd2f920486e None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:53.550521 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 349/687] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:34:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:53.552105 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9760c-28f7-4f4f-bf58-cf2c157e0e83 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:53.574990 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac77d0-f1e1-4f07-ab71-9f417b0fb4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:53.575717 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac77d0-f1e1-4f07-ab71-9f417b0fb4be None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:53.576002 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac77d0-f1e1-4f07-ab71-9f417b0fb4be None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:53.576286 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac77d0-f1e1-4f07-ab71-9f417b0fb4be None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:53.577072 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac77d0-f1e1-4f07-ab71-9f417b0fb4be None admin] RBAC: Adding query filter params (name=admin) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:53.577376 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac77d0-f1e1-4f07-ab71-9f417b0fb4be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:53.579604 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac77d0-f1e1-4f07-ab71-9f417b0fb4be None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:53.592687 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 339/688] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:34:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:53.593670 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac77d0-f1e1-4f07-ab71-9f417b0fb4be None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:53.666975 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9760c-28f7-4f4f-bf58-cf2c157e0e83 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:53.702076 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac77d0-f1e1-4f07-ab71-9f417b0fb4be None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:53.707304 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f21b2-84bd-4ba8-b0e3-e991871edc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:53.708972 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f21b2-84bd-4ba8-b0e3-e991871edc22 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:53.709486 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f21b2-84bd-4ba8-b0e3-e991871edc22 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:53.710096 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f21b2-84bd-4ba8-b0e3-e991871edc22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:53.717554 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:53.724934 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f21b2-84bd-4ba8-b0e3-e991871edc22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:53.727042 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f21b2-84bd-4ba8-b0e3-e991871edc22 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:53.739737 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa5cbd-fc85-43ac-b47e-dd803e599afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:53.740404 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa5cbd-fc85-43ac-b47e-dd803e599afc None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:53.740690 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa5cbd-fc85-43ac-b47e-dd803e599afc None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:53.741002 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa5cbd-fc85-43ac-b47e-dd803e599afc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:53.741844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa5cbd-fc85-43ac-b47e-dd803e599afc None admin] RBAC: Adding query filter params (role.id=62e9fa78a7c44a16bb4540d3e8d7044d, scope.project.id=59333e4f593346e184877d6bd9d22026, user.id=7658ffde740d446682c1876662815987) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:53.742161 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa5cbd-fc85-43ac-b47e-dd803e599afc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:53.743866 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa5cbd-fc85-43ac-b47e-dd803e599afc None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:53.748355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-795f21b2-84bd-4ba8-b0e3-e991871edc22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:53.748355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:53.748355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:53.748355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:53.748355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:53.748355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:53.748355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:53.748355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:53.748355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:53.748355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:53.748355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:53.748355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:53.748355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:53.748355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:53.748355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:53.749247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:53.749247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:53.749247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:53.749247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:53.749247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:53.749247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:53.749247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:53.749247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:53.749247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:53.749247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:53.749247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:53.749247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:53.749247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:53.749247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:53.749247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:53.749247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:53.749247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:53.750216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:53.750216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:53.750216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:53.750216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:53.750897 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 340/689] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:34:53 2022] GET /identity/v3/role_assignments?user.id=7658ffde740d446682c1876662815987&scope.project.id=59333e4f593346e184877d6bd9d22026&role.id=62e9fa78a7c44a16bb4540d3e8d7044d => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:53.751662 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa5cbd-fc85-43ac-b47e-dd803e599afc None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:53.752958 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 350/690] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:53.761048 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f21b2-84bd-4ba8-b0e3-e991871edc22 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:53.819382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:53.866778 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa5cbd-fc85-43ac-b47e-dd803e599afc None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:53.878700 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f21b2-84bd-4ba8-b0e3-e991871edc22 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:53.908719 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aaac1b-d664-4f86-a248-540c297fb9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:53.908719 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aaac1b-d664-4f86-a248-540c297fb9e6 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:53.908719 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aaac1b-d664-4f86-a248-540c297fb9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:53.908719 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aaac1b-d664-4f86-a248-540c297fb9e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:53.921612 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aaac1b-d664-4f86-a248-540c297fb9e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:53.921612 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aaac1b-d664-4f86-a248-540c297fb9e6 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:53.939859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-49aaac1b-d664-4f86-a248-540c297fb9e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:53.939859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:53.939859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:53.939859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:53.939859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:53.939859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:53.939859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:53.939859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:53.939859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:53.939859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:53.939859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:53.939859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:53.939859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:53.939859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:53.939859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:53.940883 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:53.940883 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:53.940883 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:53.940883 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:53.940883 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:53.940883 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:53.940883 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:53.940883 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:53.940883 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:53.940883 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:53.940883 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:53.940883 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:53.940883 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:53.940883 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:53.940883 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:53.940883 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:53.940883 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:53.941881 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:53.941881 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:53.941881 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:53.941881 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:53.941881 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 341/691] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:53.947597 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aaac1b-d664-4f86-a248-540c297fb9e6 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:53.967309 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e043be-567c-4a17-8fef-2cf7dbe849a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:53.967999 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e043be-567c-4a17-8fef-2cf7dbe849a6 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:53.968292 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e043be-567c-4a17-8fef-2cf7dbe849a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:53.968575 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e043be-567c-4a17-8fef-2cf7dbe849a6 None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:53.969322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e043be-567c-4a17-8fef-2cf7dbe849a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:53.969632 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e043be-567c-4a17-8fef-2cf7dbe849a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:53.971031 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e043be-567c-4a17-8fef-2cf7dbe849a6 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:53.985330 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 351/692] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:34:53.990175 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:53.991690 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e043be-567c-4a17-8fef-2cf7dbe849a6 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:54.060110 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aaac1b-d664-4f86-a248-540c297fb9e6 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:54.095592 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21250731-4bee-4bf9-ac28-d5ffd1397822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:54.096279 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21250731-4bee-4bf9-ac28-d5ffd1397822 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:54.096571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21250731-4bee-4bf9-ac28-d5ffd1397822 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:54.096852 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21250731-4bee-4bf9-ac28-d5ffd1397822 None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:54.097593 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21250731-4bee-4bf9-ac28-d5ffd1397822 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:54.097902 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21250731-4bee-4bf9-ac28-d5ffd1397822 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:54.100125 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21250731-4bee-4bf9-ac28-d5ffd1397822 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:54.108833 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e043be-567c-4a17-8fef-2cf7dbe849a6 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:54.113128 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 342/693] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:54.122779 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21250731-4bee-4bf9-ac28-d5ffd1397822 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:54.124788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:54.162688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8389bc-2825-4ff5-aef5-9d9499518c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:54.164606 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8389bc-2825-4ff5-aef5-9d9499518c29 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:54.165119 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8389bc-2825-4ff5-aef5-9d9499518c29 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:54.165658 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8389bc-2825-4ff5-aef5-9d9499518c29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:54.178250 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8389bc-2825-4ff5-aef5-9d9499518c29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:54.179990 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8389bc-2825-4ff5-aef5-9d9499518c29 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:54.200724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-fc8389bc-2825-4ff5-aef5-9d9499518c29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:34:54.200724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:54.200724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:54.200724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:54.200724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:54.200724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:54.200724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:54.200724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:54.200724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:54.200724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:54.200724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:54.200724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:54.200724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:34:54.200724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:34:54.200724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:34:54.201853 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:34:54.201853 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:54.201853 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:54.201853 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:54.201853 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:54.201853 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:54.201853 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:54.201853 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:54.201853 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:54.201853 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:54.201853 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:54.201853 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:54.201853 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:54.201853 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:54.201853 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:54.201853 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:54.201853 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:54.202903 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:34:54.202903 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:34:54.202903 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:34:54.202903 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:34:54.202903 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:34:54.202903 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:34:54.202903 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:34:54.202903 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:54.207131 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 352/694] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:34:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:54.211656 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8389bc-2825-4ff5-aef5-9d9499518c29 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:54.249047 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:54.262614 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21250731-4bee-4bf9-ac28-d5ffd1397822 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:54.300131 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c333915-7092-4544-b498-244325e2f326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:54.301112 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c333915-7092-4544-b498-244325e2f326 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:54.301496 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c333915-7092-4544-b498-244325e2f326 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:54.301879 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c333915-7092-4544-b498-244325e2f326 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:54.310424 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c333915-7092-4544-b498-244325e2f326 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:54.312437 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c333915-7092-4544-b498-244325e2f326 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:54.328504 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-0c333915-7092-4544-b498-244325e2f326 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 14:34:54.328504 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:54.328504 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:54.328504 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:54.328504 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:54.328504 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:54.328504 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:54.328504 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:54.328504 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:54.328504 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:54.328504 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:54.328504 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:54.328504 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:34:54.328504 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:34:54.328504 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:34:54.328504 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:34:54.329945 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:54.329945 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:54.329945 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:34:54.329945 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:54.329945 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:34:54.329945 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:54.329945 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:54.329945 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:54.329945 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:54.329945 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:54.329945 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:54.329945 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:54.329945 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:54.329945 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:54.329945 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:54.329945 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:54.329945 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:54.329945 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:54.331407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:54.331407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:54.331407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:34:54.331407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:34:54.331407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:34:54.331407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:34:54.331407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:34:54.331407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:34:54.331407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 14:34:54.331407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:54.333158 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 343/695] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Jul 20 14:34:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:54.341682 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c333915-7092-4544-b498-244325e2f326 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:54.344828 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8389bc-2825-4ff5-aef5-9d9499518c29 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:54.384472 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a409cbd-ab4b-4faa-853b-46b84668981e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:54.385533 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a409cbd-ab4b-4faa-853b-46b84668981e None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:54.386041 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a409cbd-ab4b-4faa-853b-46b84668981e None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:54.386619 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a409cbd-ab4b-4faa-853b-46b84668981e None admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:54.387739 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a409cbd-ab4b-4faa-853b-46b84668981e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:54.388267 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a409cbd-ab4b-4faa-853b-46b84668981e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:54.390025 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a409cbd-ab4b-4faa-853b-46b84668981e None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:54.403930 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 353/696] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:34:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:54.409270 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a409cbd-ab4b-4faa-853b-46b84668981e None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:54.455436 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:54.564485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c333915-7092-4544-b498-244325e2f326 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:54.600349 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a409cbd-ab4b-4faa-853b-46b84668981e None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:54.601671 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f3d7d-5721-4eed-8830-e17c01c909f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:54.602459 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f3d7d-5721-4eed-8830-e17c01c909f0 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:54.602742 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f3d7d-5721-4eed-8830-e17c01c909f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:54.603214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f3d7d-5721-4eed-8830-e17c01c909f0 None admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:54.604078 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9f3d7d-5721-4eed-8830-e17c01c909f0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:54.604388 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9f3d7d-5721-4eed-8830-e17c01c909f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:54.605852 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9f3d7d-5721-4eed-8830-e17c01c909f0 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:54.624828 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 344/697] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Jul 20 14:34:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:54.629913 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f3d7d-5721-4eed-8830-e17c01c909f0 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:54.651453 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:54.671813 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3630ed8-ad4e-413e-ba59-328b29ffe554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:54.674070 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3630ed8-ad4e-413e-ba59-328b29ffe554 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:54.674604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3630ed8-ad4e-413e-ba59-328b29ffe554 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:54.675102 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3630ed8-ad4e-413e-ba59-328b29ffe554 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:54.676362 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3630ed8-ad4e-413e-ba59-328b29ffe554 None admin] RBAC: Adding query filter params (role.id=62e9fa78a7c44a16bb4540d3e8d7044d, scope.project.id=59333e4f593346e184877d6bd9d22026, user.id=37f4844988474d7a91133d77906611a5) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:54.679073 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3630ed8-ad4e-413e-ba59-328b29ffe554 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:54.681654 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3630ed8-ad4e-413e-ba59-328b29ffe554 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:54.693691 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 354/698] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:34:54 2022] GET /identity/v3/role_assignments?user.id=37f4844988474d7a91133d77906611a5&scope.project.id=59333e4f593346e184877d6bd9d22026&role.id=62e9fa78a7c44a16bb4540d3e8d7044d => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:54.773238 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f3d7d-5721-4eed-8830-e17c01c909f0 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:54.811670 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f4827-b0ee-4a89-8810-4940c7c42fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:54.812741 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f4827-b0ee-4a89-8810-4940c7c42fb0 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:54.813241 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f4827-b0ee-4a89-8810-4940c7c42fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:54.813724 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f4827-b0ee-4a89-8810-4940c7c42fb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:54.856477 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f9765-1c20-49c2-942d-39a31109535f None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:54.857422 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f9765-1c20-49c2-942d-39a31109535f None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:54.857929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f9765-1c20-49c2-942d-39a31109535f None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:54.859859 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 355/699] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:54.865925 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650cde6f-963a-422e-8f9c-5017492ca78a None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:54.866551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650cde6f-963a-422e-8f9c-5017492ca78a None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:54.867038 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650cde6f-963a-422e-8f9c-5017492ca78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:54.885577 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-650cde6f-963a-422e-8f9c-5017492ca78a None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:54.887225 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-650cde6f-963a-422e-8f9c-5017492ca78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:54.938348 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f4827-b0ee-4a89-8810-4940c7c42fb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:54.940791 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f4827-b0ee-4a89-8810-4940c7c42fb0 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:54.950246 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:54.983662 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 356/700] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 14:34:55.007296 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650cde6f-963a-422e-8f9c-5017492ca78a None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:55.032874 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:55.053903 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 345/701] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:34:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 14:34:55.059607 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f4827-b0ee-4a89-8810-4940c7c42fb0 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:55.117835 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650cde6f-963a-422e-8f9c-5017492ca78a None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:55.157083 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d146f0-803c-4fda-b80b-af872036bd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:55.158097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d146f0-803c-4fda-b80b-af872036bd7f None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:55.158452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d146f0-803c-4fda-b80b-af872036bd7f None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:55.158791 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d146f0-803c-4fda-b80b-af872036bd7f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:55.160343 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d146f0-803c-4fda-b80b-af872036bd7f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:55.162061 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d146f0-803c-4fda-b80b-af872036bd7f None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:55.179895 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-22d146f0-803c-4fda-b80b-af872036bd7f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 14:34:55.179895 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:55.179895 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 14:34:55.179895 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 14:34:55.179895 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 14:34:55.179895 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 14:34:55.179895 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 14:34:55.179895 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:34:55.179895 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 14:34:55.179895 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:55.179895 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 14:34:55.179895 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:55.179895 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:55.179895 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:55.179895 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:55.179895 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:55.180998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:55.180998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:55.180998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:55.180998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:55.180998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:55.180998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:55.180998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:55.180998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 14:34:55.180998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 14:34:55.180998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 14:34:55.180998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 14:34:55.180998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:55.180998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:55.180998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:55.180998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:55.180998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:55.181951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:55.181951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:55.181951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:55.181951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:55.181951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:55.181951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:55.181951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:55.181951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:55.181951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:55.181951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:55.181951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:55.181951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 14:34:55.181951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:34:55.181951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 14:34:55.181951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:55.181951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 357/702] 10.222.0.15 () {56 vars in 1129 bytes} [Wed Jul 20 14:34:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:34:55.182808 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f4827-b0ee-4a89-8810-4940c7c42fb0 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:55.189454 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d146f0-803c-4fda-b80b-af872036bd7f None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:55.196729 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:55.224636 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf78fa-a48c-408f-9c6c-e69521314ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:55.225525 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf78fa-a48c-408f-9c6c-e69521314ca6 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:55.225808 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf78fa-a48c-408f-9c6c-e69521314ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:55.226083 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf78fa-a48c-408f-9c6c-e69521314ca6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:55.237683 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caf78fa-a48c-408f-9c6c-e69521314ca6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:55.239457 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caf78fa-a48c-408f-9c6c-e69521314ca6 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:55.259390 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-3caf78fa-a48c-408f-9c6c-e69521314ca6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:55.259390 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:55.259390 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:55.259390 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:55.259390 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:55.259390 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:55.259390 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:55.259390 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:55.259390 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:55.259390 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:55.259390 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:55.259390 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:55.259390 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:55.259390 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:55.259390 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:55.260346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:55.260346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:55.260346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:55.260346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:55.260346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:55.260346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:55.260346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:55.260346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:55.260346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:55.260346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:55.260346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:55.260346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:55.260346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:55.260346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:55.260346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:55.260346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:55.260346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:55.261236 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:55.261236 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:55.261236 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:55.261236 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:55.263699 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 346/703] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:55.265150 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d584dd-5de6-4480-838e-2093c944bc54 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:55.265627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d584dd-5de6-4480-838e-2093c944bc54 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:55.266070 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d584dd-5de6-4480-838e-2093c944bc54 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:55.267766 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 347/704] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:55.272977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d584dd-5de6-4480-838e-2093c944bc54 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:55.304280 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:55.323991 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d146f0-803c-4fda-b80b-af872036bd7f None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:55.362535 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e333ee-6687-45db-8548-ac3f1732190a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:55.363334 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e333ee-6687-45db-8548-ac3f1732190a None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:55.363542 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e333ee-6687-45db-8548-ac3f1732190a None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:55.363825 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e333ee-6687-45db-8548-ac3f1732190a None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:55.364557 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e333ee-6687-45db-8548-ac3f1732190a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:55.364937 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e333ee-6687-45db-8548-ac3f1732190a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:55.366473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e333ee-6687-45db-8548-ac3f1732190a None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:55.371714 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 358/705] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Jul 20 14:34:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:55.372916 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe053a-4a0d-49cd-b743-1ced51420ece None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:55.373205 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe053a-4a0d-49cd-b743-1ced51420ece None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:55.373495 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe053a-4a0d-49cd-b743-1ced51420ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:55.397481 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-25fe053a-4a0d-49cd-b743-1ced51420ece None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:55.399123 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-25fe053a-4a0d-49cd-b743-1ced51420ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:55.426272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d584dd-5de6-4480-838e-2093c944bc54 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:55.474881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 359/706] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:55.475931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ddad7-3951-4e63-9fc7-5fcd035e12c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:55.476060 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe053a-4a0d-49cd-b743-1ced51420ece None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:55.476928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ddad7-3951-4e63-9fc7-5fcd035e12c1 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:55.477499 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ddad7-3951-4e63-9fc7-5fcd035e12c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:55.477981 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ddad7-3951-4e63-9fc7-5fcd035e12c1 None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:55.480184 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ddad7-3951-4e63-9fc7-5fcd035e12c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:55.480771 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ddad7-3951-4e63-9fc7-5fcd035e12c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:55.482494 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ddad7-3951-4e63-9fc7-5fcd035e12c1 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:55.497349 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 348/707] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:55.503319 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ddad7-3951-4e63-9fc7-5fcd035e12c1 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:55.526229 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:55.582916 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:55.612093 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe053a-4a0d-49cd-b743-1ced51420ece None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:55.629018 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ddad7-3951-4e63-9fc7-5fcd035e12c1 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:55.660516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f276ac0c-882c-4604-abd7-4f03ea5f2d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:55.661722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276ac0c-882c-4604-abd7-4f03ea5f2d5e None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:55.662509 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276ac0c-882c-4604-abd7-4f03ea5f2d5e None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:55.663222 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276ac0c-882c-4604-abd7-4f03ea5f2d5e None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:55.664099 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f276ac0c-882c-4604-abd7-4f03ea5f2d5e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:55.664548 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f276ac0c-882c-4604-abd7-4f03ea5f2d5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:55.666119 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f276ac0c-882c-4604-abd7-4f03ea5f2d5e None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:55.667215 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704f796-742d-4778-ae72-daf7bd196d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:55.668272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704f796-742d-4778-ae72-daf7bd196d38 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:55.668666 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704f796-742d-4778-ae72-daf7bd196d38 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:55.669034 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704f796-742d-4778-ae72-daf7bd196d38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:55.673999 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 360/708] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Jul 20 14:34:55 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:55.675073 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f276ac0c-882c-4604-abd7-4f03ea5f2d5e None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:55.677055 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5704f796-742d-4778-ae72-daf7bd196d38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:55.678890 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5704f796-742d-4778-ae72-daf7bd196d38 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:55.695526 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-5704f796-742d-4778-ae72-daf7bd196d38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:34:55.695526 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:55.695526 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:55.695526 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:55.695526 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:55.695526 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:55.695526 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:55.695526 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:55.695526 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:55.695526 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:55.695526 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:55.695526 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:55.695526 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:34:55.695526 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:34:55.695526 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:34:55.696718 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:34:55.696718 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:55.696718 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:55.696718 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:55.696718 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:55.696718 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:55.696718 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:55.696718 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:55.696718 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:55.696718 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:55.696718 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:55.696718 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:55.696718 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:55.696718 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:55.696718 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:55.696718 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:55.696718 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:55.697823 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:34:55.697823 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:34:55.697823 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:34:55.697823 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:34:55.697823 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:34:55.697823 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:34:55.697823 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:34:55.697823 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:55.699822 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 349/709] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:34:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:55.706115 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704f796-742d-4778-ae72-daf7bd196d38 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:55.810514 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f276ac0c-882c-4604-abd7-4f03ea5f2d5e None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:55.823766 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:55.868969 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704f796-742d-4778-ae72-daf7bd196d38 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:55.876084 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e58f8-3b20-4680-a185-f1c061c150dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:55.877643 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e58f8-3b20-4680-a185-f1c061c150dd None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:55.878123 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e58f8-3b20-4680-a185-f1c061c150dd None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:55.878590 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e58f8-3b20-4680-a185-f1c061c150dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:55.902058 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e58f8-3b20-4680-a185-f1c061c150dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:55.903638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e58f8-3b20-4680-a185-f1c061c150dd None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:55.906725 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:55.917455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c42985e-34b8-4aa9-b148-b9358eb0dea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:55.918559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c42985e-34b8-4aa9-b148-b9358eb0dea4 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:55.919091 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c42985e-34b8-4aa9-b148-b9358eb0dea4 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:55.919586 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c42985e-34b8-4aa9-b148-b9358eb0dea4 None admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:55.920581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c42985e-34b8-4aa9-b148-b9358eb0dea4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:55.921082 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c42985e-34b8-4aa9-b148-b9358eb0dea4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:55.923154 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c42985e-34b8-4aa9-b148-b9358eb0dea4 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:55.923870 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-502e58f8-3b20-4680-a185-f1c061c150dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:55.923870 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:55.923870 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:55.923870 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:55.923870 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:55.923870 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:55.923870 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:55.923870 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:55.923870 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:55.923870 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:55.923870 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:55.923870 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:55.923870 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:55.923870 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:55.923870 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:55.924787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:55.924787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:55.924787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:55.924787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:55.924787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:55.924787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:55.924787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:55.924787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:55.924787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:55.924787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:55.924787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:55.924787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:55.924787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:55.924787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:55.924787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:55.924787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:55.924787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:55.926138 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:55.926138 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:55.926138 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:55.926138 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:55.926138 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 361/710] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:55.933681 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e58f8-3b20-4680-a185-f1c061c150dd None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:55.941299 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 350/711] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:34:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:55.949391 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c42985e-34b8-4aa9-b148-b9358eb0dea4 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:56.049001 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e58f8-3b20-4680-a185-f1c061c150dd None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:56.058692 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c42985e-34b8-4aa9-b148-b9358eb0dea4 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:56.084819 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5db0d-afcb-4202-9e11-21a14af7a28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:56.085532 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5db0d-afcb-4202-9e11-21a14af7a28a None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:56.085902 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5db0d-afcb-4202-9e11-21a14af7a28a None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:56.086072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5db0d-afcb-4202-9e11-21a14af7a28a None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:56.086824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c5db0d-afcb-4202-9e11-21a14af7a28a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:56.087155 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c5db0d-afcb-4202-9e11-21a14af7a28a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:56.088718 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c5db0d-afcb-4202-9e11-21a14af7a28a None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:56.097463 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd51aa3-62cf-445d-ad3f-8cb700f01b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:56.098630 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd51aa3-62cf-445d-ad3f-8cb700f01b57 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:56.099164 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd51aa3-62cf-445d-ad3f-8cb700f01b57 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:56.099653 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd51aa3-62cf-445d-ad3f-8cb700f01b57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:56.100802 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd51aa3-62cf-445d-ad3f-8cb700f01b57 None admin] RBAC: Adding query filter params (role.id=49bd5abf255048a0abaa9538e309f516, scope.project.id=59333e4f593346e184877d6bd9d22026, user.id=5d4a107ed04348b59d20d1703dbe04eb) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:56.101308 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd51aa3-62cf-445d-ad3f-8cb700f01b57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:56.102204 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 362/712] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:56.103276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd51aa3-62cf-445d-ad3f-8cb700f01b57 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:56.110066 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 351/713] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:34:55 2022] GET /identity/v3/role_assignments?user.id=5d4a107ed04348b59d20d1703dbe04eb&scope.project.id=59333e4f593346e184877d6bd9d22026&role.id=49bd5abf255048a0abaa9538e309f516 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:56.112221 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5db0d-afcb-4202-9e11-21a14af7a28a None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:56.176991 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:56.241173 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5db0d-afcb-4202-9e11-21a14af7a28a None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:56.290349 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25305dd4-e77b-438f-a4f1-76bbbbf6b4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:56.291627 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25305dd4-e77b-438f-a4f1-76bbbbf6b4b8 None admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:56.292160 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25305dd4-e77b-438f-a4f1-76bbbbf6b4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:56.292635 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25305dd4-e77b-438f-a4f1-76bbbbf6b4b8 None admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:56.293613 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25305dd4-e77b-438f-a4f1-76bbbbf6b4b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:56.295376 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25305dd4-e77b-438f-a4f1-76bbbbf6b4b8 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:56.327340 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 363/714] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jul 20 14:34:56 2022] POST /identity/v3/users => generated 254 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:34:56.584122 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816629b1-c318-43f7-acfa-df5a0c3a2290 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:56.584895 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816629b1-c318-43f7-acfa-df5a0c3a2290 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:56.585393 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816629b1-c318-43f7-acfa-df5a0c3a2290 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:56.587013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 352/715] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:56.592851 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca446af-efe3-48a8-bf06-17d7538bdecd None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:56.593442 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca446af-efe3-48a8-bf06-17d7538bdecd None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:56.593917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca446af-efe3-48a8-bf06-17d7538bdecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:56.614004 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-7ca446af-efe3-48a8-bf06-17d7538bdecd None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:56.615538 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-7ca446af-efe3-48a8-bf06-17d7538bdecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:56.688819 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 364/716] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:56.710462 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816629b1-c318-43f7-acfa-df5a0c3a2290 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:56.714478 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:56.822254 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816629b1-c318-43f7-acfa-df5a0c3a2290 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:56.857579 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f941f7-9b3e-4317-ace0-6b5137e39e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:56.858760 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f941f7-9b3e-4317-ace0-6b5137e39e18 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:56.859274 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f941f7-9b3e-4317-ace0-6b5137e39e18 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:56.859742 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f941f7-9b3e-4317-ace0-6b5137e39e18 None admin] PATH_INFO: `/v3/services/network` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:56.860587 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f941f7-9b3e-4317-ace0-6b5137e39e18 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:56.862162 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f941f7-9b3e-4317-ace0-6b5137e39e18 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:56.876947 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-a0f941f7-9b3e-4317-ace0-6b5137e39e18 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 14:34:56.876947 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:56.876947 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 14:34:56.876947 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 14:34:56.876947 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 14:34:56.876947 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 14:34:56.876947 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 14:34:56.876947 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:34:56.876947 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 14:34:56.876947 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:56.876947 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 14:34:56.876947 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:56.876947 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:56.876947 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:56.876947 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:56.876947 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:56.876947 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:56.877962 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:56.877962 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:56.877962 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:56.877962 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:56.877962 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:56.877962 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:56.877962 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 14:34:56.877962 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 14:34:56.877962 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 14:34:56.877962 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 14:34:56.877962 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:56.877962 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:56.877962 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:56.877962 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:56.877962 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:56.877962 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:56.878827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:56.878827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:56.878827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:56.878827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:56.878827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:56.878827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:56.878827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:56.878827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:56.878827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:56.878827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:56.878827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 14:34:56.878827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:34:56.878827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 14:34:56.878827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:56.881798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 353/717] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:34:56 2022] GET /identity/v3/services/network => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:56.892277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca446af-efe3-48a8-bf06-17d7538bdecd None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:56.948675 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:57.023698 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca446af-efe3-48a8-bf06-17d7538bdecd None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:57.066549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c8f0ed-fae6-4d1b-b714-892f6cefea4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:57.067258 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8f0ed-fae6-4d1b-b714-892f6cefea4c None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:57.067542 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8f0ed-fae6-4d1b-b714-892f6cefea4c None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:57.067818 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8f0ed-fae6-4d1b-b714-892f6cefea4c None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:57.068536 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c8f0ed-fae6-4d1b-b714-892f6cefea4c None admin] RBAC: Adding query filter params (name=network) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:57.068876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c8f0ed-fae6-4d1b-b714-892f6cefea4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:57.070314 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c8f0ed-fae6-4d1b-b714-892f6cefea4c None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:57.075649 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 365/718] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Jul 20 14:34:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:57.081311 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f941f7-9b3e-4317-ace0-6b5137e39e18 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:57.170721 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:57.221360 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f941f7-9b3e-4317-ace0-6b5137e39e18 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:57.263677 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d29b0-6b22-4d1e-b382-7ba50bca50cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:57.264360 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d29b0-6b22-4d1e-b382-7ba50bca50cf None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:57.264643 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d29b0-6b22-4d1e-b382-7ba50bca50cf None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:57.264919 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d29b0-6b22-4d1e-b382-7ba50bca50cf None admin] PATH_INFO: `/v3/services` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:57.265637 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d29b0-6b22-4d1e-b382-7ba50bca50cf None admin] RBAC: Adding query filter params (type=network) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:57.265943 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d29b0-6b22-4d1e-b382-7ba50bca50cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:57.267372 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d29b0-6b22-4d1e-b382-7ba50bca50cf None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:57.273792 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 354/719] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Jul 20 14:34:57 2022] GET /identity/v3/services?type=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:57.426580 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d36c48-abdc-49e9-a49b-4eab5a879038 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:57.426932 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d36c48-abdc-49e9-a49b-4eab5a879038 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:57.427074 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d36c48-abdc-49e9-a49b-4eab5a879038 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:57.428770 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 366/720] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:57.434761 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8c8d7-b7e8-46de-9d2a-8a901a11e015 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:57.435118 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8c8d7-b7e8-46de-9d2a-8a901a11e015 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:57.435356 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8c8d7-b7e8-46de-9d2a-8a901a11e015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:57.459569 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-10a8c8d7-b7e8-46de-9d2a-8a901a11e015 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:57.460896 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-10a8c8d7-b7e8-46de-9d2a-8a901a11e015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:57.561759 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 355/721] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 14:34:57.585656 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d36c48-abdc-49e9-a49b-4eab5a879038 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:57.737462 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d36c48-abdc-49e9-a49b-4eab5a879038 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:57.745105 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:57.784705 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81f0b6-b713-48c6-a9d6-e98817576368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:57.785512 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81f0b6-b713-48c6-a9d6-e98817576368 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:57.785797 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81f0b6-b713-48c6-a9d6-e98817576368 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:57.786075 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81f0b6-b713-48c6-a9d6-e98817576368 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:57.805842 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b81f0b6-b713-48c6-a9d6-e98817576368 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:57.813666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-9b81f0b6-b713-48c6-a9d6-e98817576368 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:34:57.813666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:57.813666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:57.813666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:57.813666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:57.813666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:57.813666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:57.813666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:57.813666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:57.813666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:57.813666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:57.813666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:57.813666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:57.813666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:57.813666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:57.813666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:57.814603 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:57.814603 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:57.814603 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:57.814603 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:57.814603 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:57.814603 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:57.814603 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:57.814603 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:57.814603 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:57.814603 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:57.814603 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:57.814603 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:57.814603 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:57.814603 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:57.814603 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:57.814603 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:57.816039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:57.816039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:57.816039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:57.816039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:57.816039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:57.816039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:57.816039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:57.816039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:57.816039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:57.816039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:57.816039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:34:57.816039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:57.816039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 367/722] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:34:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:57.820746 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8c8d7-b7e8-46de-9d2a-8a901a11e015 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:57.892305 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:57.946773 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8c8d7-b7e8-46de-9d2a-8a901a11e015 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:57.991539 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1948eb4-0b62-42ba-b7b8-08960681dbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:57.992600 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1948eb4-0b62-42ba-b7b8-08960681dbd4 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:57.993085 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1948eb4-0b62-42ba-b7b8-08960681dbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:57.993568 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1948eb4-0b62-42ba-b7b8-08960681dbd4 None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:57.994521 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1948eb4-0b62-42ba-b7b8-08960681dbd4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:57.995021 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1948eb4-0b62-42ba-b7b8-08960681dbd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:57.996657 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1948eb4-0b62-42ba-b7b8-08960681dbd4 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:58.006511 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 356/723] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:34:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:58.011882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81f0b6-b713-48c6-a9d6-e98817576368 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:58.174495 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81f0b6-b713-48c6-a9d6-e98817576368 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:58.222733 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789be05f-18cf-4d3e-b89a-d0392ef2b098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:58.223573 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789be05f-18cf-4d3e-b89a-d0392ef2b098 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:58.223863 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789be05f-18cf-4d3e-b89a-d0392ef2b098 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:58.224145 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789be05f-18cf-4d3e-b89a-d0392ef2b098 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:58.239497 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789be05f-18cf-4d3e-b89a-d0392ef2b098 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:58.241007 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789be05f-18cf-4d3e-b89a-d0392ef2b098 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:58.258776 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-789be05f-18cf-4d3e-b89a-d0392ef2b098 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:58.258776 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:58.258776 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:58.258776 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:58.258776 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:58.258776 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:58.258776 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:58.258776 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:58.258776 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:58.258776 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:58.258776 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:58.258776 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:58.258776 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:58.258776 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:58.258776 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:58.259722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:58.259722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:58.259722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:58.259722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:58.259722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:58.259722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:58.259722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:58.259722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:58.259722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:58.259722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:58.259722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:58.259722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:58.259722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:58.259722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:58.259722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:58.259722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:58.259722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:58.260745 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:58.260745 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:58.260745 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:58.260745 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:58.270150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 368/724] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:58.275724 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1948eb4-0b62-42ba-b7b8-08960681dbd4 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:58.397702 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1948eb4-0b62-42ba-b7b8-08960681dbd4 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:58.432807 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:58.446410 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021684e2-fc81-4858-8b5d-12b86bfb9584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:58.447119 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021684e2-fc81-4858-8b5d-12b86bfb9584 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:58.447408 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021684e2-fc81-4858-8b5d-12b86bfb9584 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:58.447775 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021684e2-fc81-4858-8b5d-12b86bfb9584 None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:58.448502 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021684e2-fc81-4858-8b5d-12b86bfb9584 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:58.448804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021684e2-fc81-4858-8b5d-12b86bfb9584 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:58.450177 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021684e2-fc81-4858-8b5d-12b86bfb9584 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:58.466770 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 357/725] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:58.468142 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789be05f-18cf-4d3e-b89a-d0392ef2b098 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:58.520391 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:58.641294 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789be05f-18cf-4d3e-b89a-d0392ef2b098 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:58.643004 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a7ba1-ac6c-45b2-959e-4bedd4fc247c None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:58.643427 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a7ba1-ac6c-45b2-959e-4bedd4fc247c None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:58.643627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a7ba1-ac6c-45b2-959e-4bedd4fc247c None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:58.646272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 358/726] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:58.652500 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff9cb7-a3c2-4b96-b6f4-57f3097e47a0 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:58.652798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff9cb7-a3c2-4b96-b6f4-57f3097e47a0 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:58.653080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff9cb7-a3c2-4b96-b6f4-57f3097e47a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:58.669735 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-52ff9cb7-a3c2-4b96-b6f4-57f3097e47a0 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:58.670928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-52ff9cb7-a3c2-4b96-b6f4-57f3097e47a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:58.695716 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d63ba-cb09-427b-8c50-3720b583cede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:58.696536 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d63ba-cb09-427b-8c50-3720b583cede None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:58.697206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d63ba-cb09-427b-8c50-3720b583cede None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:58.697485 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d63ba-cb09-427b-8c50-3720b583cede None admin] PATH_INFO: `/v3/users/swift` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:58.707894 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256d63ba-cb09-427b-8c50-3720b583cede None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:58.709790 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256d63ba-cb09-427b-8c50-3720b583cede None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:58.730165 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-256d63ba-cb09-427b-8c50-3720b583cede None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 14:34:58.730165 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:58.730165 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:58.730165 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:58.730165 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:58.730165 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:58.730165 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:58.730165 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:58.730165 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:58.730165 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:58.730165 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:58.730165 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:58.730165 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:34:58.730165 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:34:58.730165 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:34:58.730165 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:34:58.731218 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:58.731218 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:58.731218 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:34:58.731218 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:58.731218 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:34:58.731218 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:58.731218 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:58.731218 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:58.731218 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:58.731218 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:58.731218 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:58.731218 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:58.731218 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:58.731218 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:58.731218 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:58.731218 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:58.731218 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:58.731218 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:58.732187 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:58.732187 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:58.732187 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:34:58.732187 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:34:58.732187 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:34:58.732187 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:34:58.732187 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:34:58.732187 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:34:58.732187 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 14:34:58.732187 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:58.732187 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 369/727] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:34:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:58.739072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3697457e-03f4-4ba8-b36b-881b2dc5b7b2 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:58.739072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3697457e-03f4-4ba8-b36b-881b2dc5b7b2 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:58.739072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3697457e-03f4-4ba8-b36b-881b2dc5b7b2 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:58.739072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 370/728] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:58.739072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3697457e-03f4-4ba8-b36b-881b2dc5b7b2 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:58.752257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 359/729] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:58.753587 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63c4d9-a1b3-4037-929a-fb02d6fee237 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:58.753880 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63c4d9-a1b3-4037-929a-fb02d6fee237 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:58.754158 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63c4d9-a1b3-4037-929a-fb02d6fee237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:58.783564 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-9e63c4d9-a1b3-4037-929a-fb02d6fee237 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:58.784893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-9e63c4d9-a1b3-4037-929a-fb02d6fee237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:58.838122 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:58.856886 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 360/730] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:58.857747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63c4d9-a1b3-4037-929a-fb02d6fee237 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:58.872191 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3697457e-03f4-4ba8-b36b-881b2dc5b7b2 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:58.901660 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:58.923480 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a9f67-501e-47d6-bf6e-16f83710ba91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:58.924698 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a9f67-501e-47d6-bf6e-16f83710ba91 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:58.925262 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a9f67-501e-47d6-bf6e-16f83710ba91 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:58.925740 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a9f67-501e-47d6-bf6e-16f83710ba91 None admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:58.926933 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a9f67-501e-47d6-bf6e-16f83710ba91 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:58.927489 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a9f67-501e-47d6-bf6e-16f83710ba91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:58.929443 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a9f67-501e-47d6-bf6e-16f83710ba91 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:58.949782 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 371/731] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Jul 20 14:34:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:58.950891 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a9f67-501e-47d6-bf6e-16f83710ba91 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:58.967099 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63c4d9-a1b3-4037-929a-fb02d6fee237 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:58.992458 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bbdfaf-efd9-431c-87fb-fc242ae77406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:58.993238 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbdfaf-efd9-431c-87fb-fc242ae77406 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:58.993523 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbdfaf-efd9-431c-87fb-fc242ae77406 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:58.993795 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbdfaf-efd9-431c-87fb-fc242ae77406 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:59.004429 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bbdfaf-efd9-431c-87fb-fc242ae77406 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:59.012079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-71bbdfaf-efd9-431c-87fb-fc242ae77406 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:34:59.012079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:59.012079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:59.012079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:59.012079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:59.012079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:59.012079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:59.012079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:59.012079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:59.012079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:59.012079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:59.012079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:59.012079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:59.012079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:59.012079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:59.012079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:59.013042 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:59.013042 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:59.013042 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:59.013042 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:59.013042 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:59.013042 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:59.013042 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:59.013042 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:59.013042 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:59.013042 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:59.013042 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:59.013042 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:59.013042 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:59.013042 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:59.013042 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:59.013042 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:59.013894 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:59.013894 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:59.013894 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:59.013894 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:59.013894 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:59.013894 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:59.013894 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:59.013894 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:59.013894 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:59.013894 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:59.013894 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:34:59.013894 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:59.018072 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 361/732] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:34:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:59.018988 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bbdfaf-efd9-431c-87fb-fc242ae77406 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:59.054377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a9f67-501e-47d6-bf6e-16f83710ba91 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:59.089431 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3c050-15e3-42fb-ac7f-f863aef93fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:59.090118 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3c050-15e3-42fb-ac7f-f863aef93fa6 None admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:59.090400 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3c050-15e3-42fb-ac7f-f863aef93fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:59.090674 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3c050-15e3-42fb-ac7f-f863aef93fa6 None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:59.091448 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c3c050-15e3-42fb-ac7f-f863aef93fa6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:59.092838 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c3c050-15e3-42fb-ac7f-f863aef93fa6 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:59.099109 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:59.118462 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 372/733] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Jul 20 14:34:58 2022] POST /identity/v3/services => generated 251 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:34:59.119947 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3c050-15e3-42fb-ac7f-f863aef93fa6 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:59.140734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bbdfaf-efd9-431c-87fb-fc242ae77406 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:59.143276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:59.209203 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78912b35-65fa-4583-8566-f2d5bbdc7dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:59.210080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78912b35-65fa-4583-8566-f2d5bbdc7dc6 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:59.210459 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78912b35-65fa-4583-8566-f2d5bbdc7dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:59.210826 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78912b35-65fa-4583-8566-f2d5bbdc7dc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:59.261454 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3c050-15e3-42fb-ac7f-f863aef93fa6 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:59.303562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78912b35-65fa-4583-8566-f2d5bbdc7dc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:59.306276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78912b35-65fa-4583-8566-f2d5bbdc7dc6 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:59.306388 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376911c-f5bd-4a7e-9d0b-86fe095a294f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:59.306388 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376911c-f5bd-4a7e-9d0b-86fe095a294f None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:59.306569 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376911c-f5bd-4a7e-9d0b-86fe095a294f None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:59.306823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376911c-f5bd-4a7e-9d0b-86fe095a294f None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:59.307660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f376911c-f5bd-4a7e-9d0b-86fe095a294f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:59.307972 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f376911c-f5bd-4a7e-9d0b-86fe095a294f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:59.309346 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f376911c-f5bd-4a7e-9d0b-86fe095a294f None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:59.319078 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 373/734] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:34:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:34:59.320530 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571f447-6fb3-40eb-843f-b59ab52a087f None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:59.320884 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571f447-6fb3-40eb-843f-b59ab52a087f None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:59.321229 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571f447-6fb3-40eb-843f-b59ab52a087f None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:59.322817 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 374/735] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:59.327792 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571f447-6fb3-40eb-843f-b59ab52a087f None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:59.406027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:59.423412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:59.427969 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 362/736] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:34:59 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 14:34:59.430294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d81ef-41c7-4902-9786-bdd907fa7eb6 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:59.430888 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d81ef-41c7-4902-9786-bdd907fa7eb6 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:59.431198 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d81ef-41c7-4902-9786-bdd907fa7eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:59.453054 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-ba6d81ef-41c7-4902-9786-bdd907fa7eb6 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:34:59.454245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-ba6d81ef-41c7-4902-9786-bdd907fa7eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:34:59.455738 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571f447-6fb3-40eb-843f-b59ab52a087f None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:59.496557 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0daae0-7923-493b-b4ae-534c10a16ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:59.497367 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0daae0-7923-493b-b4ae-534c10a16ad8 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:59.497648 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0daae0-7923-493b-b4ae-534c10a16ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:59.497925 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0daae0-7923-493b-b4ae-534c10a16ad8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:59.509131 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0daae0-7923-493b-b4ae-534c10a16ad8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:59.510697 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0daae0-7923-493b-b4ae-534c10a16ad8 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:59.526740 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 363/737] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:34:59 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:34:59.527666 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d81ef-41c7-4902-9786-bdd907fa7eb6 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:59.528917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-de0daae0-7923-493b-b4ae-534c10a16ad8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:59.528917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:59.528917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:59.528917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:59.528917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:59.528917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:59.528917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:59.528917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:59.528917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:59.528917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:59.528917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:59.528917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:59.528917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:59.528917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:59.528917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:59.529975 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:59.529975 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:59.529975 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:59.529975 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:59.529975 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:59.529975 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:59.529975 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:59.529975 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:59.529975 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:59.529975 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:59.529975 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:59.529975 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:59.529975 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:59.529975 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:59.529975 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:59.529975 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:59.529975 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:59.530892 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:59.530892 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:59.530892 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:59.530892 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:59.534911 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 375/738] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:59.536170 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0daae0-7923-493b-b4ae-534c10a16ad8 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:59.637552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d81ef-41c7-4902-9786-bdd907fa7eb6 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:59.639930 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0daae0-7923-493b-b4ae-534c10a16ad8 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:59.650868 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:59.672012 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:59.676330 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbaf85c-a163-4cc8-a2b7-2082f8e314d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:59.677094 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbaf85c-a163-4cc8-a2b7-2082f8e314d4 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:59.677436 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbaf85c-a163-4cc8-a2b7-2082f8e314d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:59.677781 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbaf85c-a163-4cc8-a2b7-2082f8e314d4 None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:59.678631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbaf85c-a163-4cc8-a2b7-2082f8e314d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:34:59.678929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbaf85c-a163-4cc8-a2b7-2082f8e314d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:59.680483 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbaf85c-a163-4cc8-a2b7-2082f8e314d4 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:59.686025 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b37ab-b83f-44e5-a2a6-4693966fd374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:59.687292 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b37ab-b83f-44e5-a2a6-4693966fd374 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:59.688273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b37ab-b83f-44e5-a2a6-4693966fd374 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:59.688764 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b37ab-b83f-44e5-a2a6-4693966fd374 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:59.697791 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 376/739] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:34:59.698722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbaf85c-a163-4cc8-a2b7-2082f8e314d4 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:59.701316 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b37ab-b83f-44e5-a2a6-4693966fd374 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:59.705522 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b37ab-b83f-44e5-a2a6-4693966fd374 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:59.730039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-5c1b37ab-b83f-44e5-a2a6-4693966fd374 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:59.730039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:59.730039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:59.730039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:59.730039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:59.730039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:59.730039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:59.730039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:59.730039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:59.730039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:59.730039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:59.730039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:59.730039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:34:59.730039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:34:59.730039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:34:59.731023 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:34:59.731023 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:59.731023 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:59.731023 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:59.731023 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:59.731023 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:59.731023 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:59.731023 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:59.731023 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:59.731023 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:59.731023 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:59.731023 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:59.731023 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:59.731023 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:59.731023 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:59.731023 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:59.731023 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:59.731960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:34:59.731960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:34:59.731960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:34:59.731960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:59.731960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 364/740] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:34:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:59.732669 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b37ab-b83f-44e5-a2a6-4693966fd374 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:59.813713 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbaf85c-a163-4cc8-a2b7-2082f8e314d4 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:59.842479 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b37ab-b83f-44e5-a2a6-4693966fd374 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:34:59.851297 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792732c-15d6-4854-852e-52c4b8880a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:59.852110 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792732c-15d6-4854-852e-52c4b8880a2d None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:59.852395 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792732c-15d6-4854-852e-52c4b8880a2d None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:59.852674 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792732c-15d6-4854-852e-52c4b8880a2d None admin] PATH_INFO: `/v3/roles/service` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:59.863556 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a792732c-15d6-4854-852e-52c4b8880a2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:34:59.871447 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-a792732c-15d6-4854-852e-52c4b8880a2d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:34:59.871447 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:59.871447 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:59.871447 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:59.871447 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:59.871447 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:59.871447 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:59.871447 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:59.871447 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:59.871447 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:59.871447 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:59.871447 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:59.871447 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:34:59.871447 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:34:59.871447 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:34:59.871447 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:34:59.872382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:34:59.872382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:34:59.872382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:34:59.872382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:34:59.872382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:59.872382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:59.872382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:59.872382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:59.872382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:59.872382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:59.872382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:59.872382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:59.872382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:59.872382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:59.872382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:59.872382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:59.873767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:59.873767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:59.873767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:59.873767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:59.873767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:34:59.873767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:34:59.873767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:34:59.873767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:34:59.873767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:34:59.873767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:34:59.873767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:34:59.873767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:34:59.873767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 377/741] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jul 20 14:34:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:59.873767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792732c-15d6-4854-852e-52c4b8880a2d None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:59.881518 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a88be37-bbb0-4290-817e-72e6996b4cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:34:59.882321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88be37-bbb0-4290-817e-72e6996b4cf1 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:59.882605 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88be37-bbb0-4290-817e-72e6996b4cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:59.882879 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88be37-bbb0-4290-817e-72e6996b4cf1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:59.891871 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a88be37-bbb0-4290-817e-72e6996b4cf1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:34:59.893264 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a88be37-bbb0-4290-817e-72e6996b4cf1 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:34:59.906592 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:59.908027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-6a88be37-bbb0-4290-817e-72e6996b4cf1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 14:34:59.908027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:34:59.908027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:34:59.908027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:34:59.908027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:34:59.908027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:34:59.908027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:34:59.908027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:34:59.908027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:34:59.908027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:34:59.908027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:34:59.908027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:34:59.908027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:34:59.908027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:34:59.908027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:34:59.908027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:34:59.909294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:34:59.909294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:34:59.909294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:34:59.909294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:59.909294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:34:59.909294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:34:59.909294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:34:59.909294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:34:59.909294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:34:59.909294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:34:59.909294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:34:59.909294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:34:59.909294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:34:59.909294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:34:59.909294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:34:59.909294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:34:59.909294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:34:59.909294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:34:59.910611 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:34:59.910611 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:34:59.910611 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:34:59.910611 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:34:59.910611 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:34:59.910611 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:34:59.910611 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:34:59.910611 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:34:59.910611 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 14:34:59.910611 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:34:59.914112 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 365/742] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Jul 20 14:34:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:34:59.915727 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbed7d-72e7-4ed5-bea5-3f89a6623aae None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:34:59.916011 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbed7d-72e7-4ed5-bea5-3f89a6623aae None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:34:59.916285 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbed7d-72e7-4ed5-bea5-3f89a6623aae None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:34:59.917858 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 366/743] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:34:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:34:59.918551 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afbed7d-72e7-4ed5-bea5-3f89a6623aae None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:34:59.965211 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:34:59.988906 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792732c-15d6-4854-852e-52c4b8880a2d None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:00.027201 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e8f48-3e1e-48f5-a9d7-0045f8bd8f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:00.028307 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e8f48-3e1e-48f5-a9d7-0045f8bd8f4a None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:00.028730 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e8f48-3e1e-48f5-a9d7-0045f8bd8f4a None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:00.029128 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e8f48-3e1e-48f5-a9d7-0045f8bd8f4a None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:00.029656 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afbed7d-72e7-4ed5-bea5-3f89a6623aae None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:00.029872 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182e8f48-3e1e-48f5-a9d7-0045f8bd8f4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:00.030187 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182e8f48-3e1e-48f5-a9d7-0045f8bd8f4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:00.031626 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182e8f48-3e1e-48f5-a9d7-0045f8bd8f4a None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:00.045683 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 378/744] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:34:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:00.052516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e8f48-3e1e-48f5-a9d7-0045f8bd8f4a None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:00.069172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f752eaa-f83d-4d1b-8071-bbd7c6745af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:00.069855 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f752eaa-f83d-4d1b-8071-bbd7c6745af8 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:00.070139 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f752eaa-f83d-4d1b-8071-bbd7c6745af8 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:00.070421 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f752eaa-f83d-4d1b-8071-bbd7c6745af8 None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:00.071199 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f752eaa-f83d-4d1b-8071-bbd7c6745af8 None admin] RBAC: Adding query filter params (name=service) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:00.071510 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f752eaa-f83d-4d1b-8071-bbd7c6745af8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:00.072905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f752eaa-f83d-4d1b-8071-bbd7c6745af8 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:00.085553 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 367/745] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jul 20 14:34:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:00.085553 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e7c27-a07b-4243-a386-6239282474f3 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:00.085553 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e7c27-a07b-4243-a386-6239282474f3 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:00.085553 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e7c27-a07b-4243-a386-6239282474f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:00.099671 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-6a3e7c27-a07b-4243-a386-6239282474f3 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:00.100893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-6a3e7c27-a07b-4243-a386-6239282474f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:00.154480 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:00.165059 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e8f48-3e1e-48f5-a9d7-0045f8bd8f4a None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:00.171377 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 368/746] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:00 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 14:35:00.172493 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3e7c27-a07b-4243-a386-6239282474f3 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:00.209269 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f7090-52da-4ade-92e0-22998a9ce4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:00.209554 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:00.210014 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f7090-52da-4ade-92e0-22998a9ce4d1 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:00.210302 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f7090-52da-4ade-92e0-22998a9ce4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:00.210629 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f7090-52da-4ade-92e0-22998a9ce4d1 None admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:00.211601 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22f7090-52da-4ade-92e0-22998a9ce4d1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:00.212029 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22f7090-52da-4ade-92e0-22998a9ce4d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:00.213462 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22f7090-52da-4ade-92e0-22998a9ce4d1 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:00.235459 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 379/747] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Jul 20 14:35:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:00.236911 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f7090-52da-4ade-92e0-22998a9ce4d1 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:00.284093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3e7c27-a07b-4243-a386-6239282474f3 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:00.318423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd89546-51b6-4b94-8146-2615ae1d8cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:00.319483 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd89546-51b6-4b94-8146-2615ae1d8cf3 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:00.319875 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd89546-51b6-4b94-8146-2615ae1d8cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:00.320257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd89546-51b6-4b94-8146-2615ae1d8cf3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:00.330059 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd89546-51b6-4b94-8146-2615ae1d8cf3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:00.331600 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd89546-51b6-4b94-8146-2615ae1d8cf3 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:00.346249 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-cdd89546-51b6-4b94-8146-2615ae1d8cf3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:35:00.346249 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:00.346249 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:00.346249 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:00.346249 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:00.346249 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:00.346249 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:00.346249 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:00.346249 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:00.346249 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:00.346249 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:00.346249 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:00.346249 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:35:00.346249 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:35:00.346249 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:35:00.347233 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:35:00.347233 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:00.347233 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:00.347233 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:00.347233 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:00.347233 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:00.347233 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:00.347233 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:00.347233 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:00.347233 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:00.347233 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:00.347233 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:00.347233 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:00.347233 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:00.347233 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:00.347233 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:00.347233 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:00.348144 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:35:00.348144 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:35:00.348144 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:35:00.348144 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:35:00.348144 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:35:00.348144 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:35:00.348144 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:35:00.348144 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:00.348144 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 369/748] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:35:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:00.348630 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd89546-51b6-4b94-8146-2615ae1d8cf3 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:00.354546 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f7090-52da-4ade-92e0-22998a9ce4d1 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:00.393862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf6709b-29ac-4c96-a83b-7238e63607b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:00.394961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6709b-29ac-4c96-a83b-7238e63607b1 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:00.395426 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6709b-29ac-4c96-a83b-7238e63607b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:00.395854 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6709b-29ac-4c96-a83b-7238e63607b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:00.406412 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf6709b-29ac-4c96-a83b-7238e63607b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:00.408158 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf6709b-29ac-4c96-a83b-7238e63607b1 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:00.418030 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:00.428695 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-9bf6709b-29ac-4c96-a83b-7238e63607b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:00.428695 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:00.428695 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:00.428695 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:00.428695 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:00.428695 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:00.428695 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:00.428695 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:00.428695 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:00.428695 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:00.428695 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:00.428695 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:00.428695 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:35:00.428695 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:35:00.428695 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:35:00.429604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:35:00.429604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:00.429604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:00.429604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:00.429604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:00.429604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:00.429604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:00.429604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:00.429604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:00.429604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:00.429604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:00.429604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:00.429604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:00.429604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:00.429604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:00.429604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:00.429604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:00.430630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:35:00.430630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:35:00.430630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:00.430630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:00.430630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 380/749] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:00.430945 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf6709b-29ac-4c96-a83b-7238e63607b1 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:00.459861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:00.470258 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd89546-51b6-4b94-8146-2615ae1d8cf3 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:00.514451 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165aae1-9859-4022-9620-8a0e6faf090d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:00.515418 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165aae1-9859-4022-9620-8a0e6faf090d None admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:00.515705 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165aae1-9859-4022-9620-8a0e6faf090d None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:00.515978 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165aae1-9859-4022-9620-8a0e6faf090d None admin] PATH_INFO: `/v3/services` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:00.516952 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165aae1-9859-4022-9620-8a0e6faf090d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:00.519067 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165aae1-9859-4022-9620-8a0e6faf090d None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:00.536592 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf6709b-29ac-4c96-a83b-7238e63607b1 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:00.536752 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 370/750] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Jul 20 14:35:00 2022] POST /identity/v3/services => generated 240 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:35:00.537485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165aae1-9859-4022-9620-8a0e6faf090d None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:00.606708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96999c-6770-434c-89db-bba921ad0693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:00.607548 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96999c-6770-434c-89db-bba921ad0693 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:00.607852 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96999c-6770-434c-89db-bba921ad0693 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:00.608126 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96999c-6770-434c-89db-bba921ad0693 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:00.626567 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce96999c-6770-434c-89db-bba921ad0693 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:00.629133 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce96999c-6770-434c-89db-bba921ad0693 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:00.653549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-ce96999c-6770-434c-89db-bba921ad0693 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:00.653549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:00.653549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:00.653549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:00.653549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:00.653549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:00.653549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:00.653549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:00.653549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:00.653549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:00.653549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:00.653549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:00.653549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:35:00.653549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:35:00.653549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:35:00.654524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:35:00.654524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:00.654524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:00.654524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:00.654524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:00.654524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:00.654524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:00.654524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:00.654524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:00.654524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:00.654524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:00.654524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:00.654524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:00.654524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:00.654524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:00.654524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:00.654524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:00.655438 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:35:00.655438 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:35:00.655438 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:00.655438 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:00.660064 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 381/751] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:00.661019 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96999c-6770-434c-89db-bba921ad0693 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:00.685885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165aae1-9859-4022-9620-8a0e6faf090d None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:00.718874 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:00.736820 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658abb77-dac6-448f-947a-8ed00979099e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:00.738323 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658abb77-dac6-448f-947a-8ed00979099e None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:00.738323 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658abb77-dac6-448f-947a-8ed00979099e None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:00.738323 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658abb77-dac6-448f-947a-8ed00979099e None admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:00.740573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658abb77-dac6-448f-947a-8ed00979099e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:00.740573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658abb77-dac6-448f-947a-8ed00979099e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:00.741855 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658abb77-dac6-448f-947a-8ed00979099e None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:00.760146 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 371/752] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:35:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:00.761009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658abb77-dac6-448f-947a-8ed00979099e None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:00.781082 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96999c-6770-434c-89db-bba921ad0693 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:00.781723 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:00.824601 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f73fd28-0c7a-4f96-a192-ae33beeca67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:00.825555 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73fd28-0c7a-4f96-a192-ae33beeca67e None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:00.825849 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73fd28-0c7a-4f96-a192-ae33beeca67e None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:00.826130 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73fd28-0c7a-4f96-a192-ae33beeca67e None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:00.827159 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f73fd28-0c7a-4f96-a192-ae33beeca67e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:00.827842 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f73fd28-0c7a-4f96-a192-ae33beeca67e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:00.829253 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f73fd28-0c7a-4f96-a192-ae33beeca67e None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:00.851029 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 382/753] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:00.853143 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f73fd28-0c7a-4f96-a192-ae33beeca67e None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:00.884111 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658abb77-dac6-448f-947a-8ed00979099e None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:00.921747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ffd871-c34c-41d4-8b05-4615e82636bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:00.922887 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffd871-c34c-41d4-8b05-4615e82636bd None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:00.923489 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffd871-c34c-41d4-8b05-4615e82636bd None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:00.923773 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffd871-c34c-41d4-8b05-4615e82636bd None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:00.924505 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ffd871-c34c-41d4-8b05-4615e82636bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:00.924818 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ffd871-c34c-41d4-8b05-4615e82636bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:00.926679 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ffd871-c34c-41d4-8b05-4615e82636bd None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:00.941940 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 372/754] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:00.947492 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ffd871-c34c-41d4-8b05-4615e82636bd None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:00.987479 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f73fd28-0c7a-4f96-a192-ae33beeca67e None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:01.000428 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:01.022699 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:01.037649 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31afc533-6f68-4131-9965-ef71fd16ba3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:01.038822 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31afc533-6f68-4131-9965-ef71fd16ba3d None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:01.039296 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31afc533-6f68-4131-9965-ef71fd16ba3d None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:01.040087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31afc533-6f68-4131-9965-ef71fd16ba3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:01.041524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31afc533-6f68-4131-9965-ef71fd16ba3d None admin] RBAC: Adding query filter params (role.id=49bd5abf255048a0abaa9538e309f516, scope.project.id=59333e4f593346e184877d6bd9d22026, user.id=37f4844988474d7a91133d77906611a5) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:01.042000 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31afc533-6f68-4131-9965-ef71fd16ba3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:01.044145 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31afc533-6f68-4131-9965-ef71fd16ba3d None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:01.059071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 383/755] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:35:00 2022] GET /identity/v3/role_assignments?user.id=37f4844988474d7a91133d77906611a5&scope.project.id=59333e4f593346e184877d6bd9d22026&role.id=49bd5abf255048a0abaa9538e309f516 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:01.059071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31afc533-6f68-4131-9965-ef71fd16ba3d None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:01.064259 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ffd871-c34c-41d4-8b05-4615e82636bd None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:01.139781 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9fa89-d619-49bf-9a2d-c0a67493de8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:01.140691 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9fa89-d619-49bf-9a2d-c0a67493de8f None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:01.140980 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9fa89-d619-49bf-9a2d-c0a67493de8f None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:01.141260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9fa89-d619-49bf-9a2d-c0a67493de8f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:01.153007 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e9fa89-d619-49bf-9a2d-c0a67493de8f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:01.153685 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e9fa89-d619-49bf-9a2d-c0a67493de8f None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:01.170652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-98e9fa89-d619-49bf-9a2d-c0a67493de8f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 14:35:01.170652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:01.170652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:01.170652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:01.170652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:01.170652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:01.170652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:01.170652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:01.170652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:01.170652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:01.170652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:01.170652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:01.170652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:35:01.170652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:35:01.170652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:35:01.170652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:35:01.171960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:01.171960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:01.171960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:35:01.171960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:35:01.171960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:35:01.171960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:35:01.171960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:01.171960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:01.171960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:01.171960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:01.171960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:01.171960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:01.171960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:01.171960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:01.171960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:01.171960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:01.171960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:01.171960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:01.173317 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:01.173317 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:01.173317 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:35:01.173317 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:35:01.173317 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:35:01.173317 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:35:01.173317 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:35:01.173317 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:35:01.173317 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 14:35:01.173317 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:01.173317 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 373/756] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Jul 20 14:35:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:01.178641 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9fa89-d619-49bf-9a2d-c0a67493de8f None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:01.194902 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31afc533-6f68-4131-9965-ef71fd16ba3d None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:01.292398 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449de1b0-3145-4b35-bf3f-780e5250f037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:01.293228 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449de1b0-3145-4b35-bf3f-780e5250f037 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:01.293514 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449de1b0-3145-4b35-bf3f-780e5250f037 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:01.293789 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449de1b0-3145-4b35-bf3f-780e5250f037 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:01.306845 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449de1b0-3145-4b35-bf3f-780e5250f037 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:01.308762 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449de1b0-3145-4b35-bf3f-780e5250f037 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:01.321941 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9fa89-d619-49bf-9a2d-c0a67493de8f None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:01.325628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-449de1b0-3145-4b35-bf3f-780e5250f037 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:35:01.325628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:01.325628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:01.325628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:01.325628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:01.325628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:01.325628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:01.325628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:01.325628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:01.325628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:01.325628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:01.325628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:01.325628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:35:01.325628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:35:01.325628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:35:01.326516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:35:01.326516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:01.326516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:01.326516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:01.326516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:01.326516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:01.326516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:01.326516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:01.326516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:01.326516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:01.326516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:01.326516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:01.326516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:01.326516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:01.326516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:01.326516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:01.326516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:01.327475 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:35:01.327475 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:35:01.327475 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:35:01.327475 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:35:01.327475 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:35:01.327475 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:35:01.327475 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:35:01.327475 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:01.329912 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 384/757] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:35:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:01.331553 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbf984-9146-4010-ba6a-487c635c83d2 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:01.332095 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbf984-9146-4010-ba6a-487c635c83d2 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:01.332572 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbf984-9146-4010-ba6a-487c635c83d2 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:01.338267 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 385/758] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:01.339502 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fbf984-9146-4010-ba6a-487c635c83d2 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:01.364619 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:01.371663 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b23637-7817-4941-9c24-5184de84957c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:01.372608 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b23637-7817-4941-9c24-5184de84957c None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:01.372897 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b23637-7817-4941-9c24-5184de84957c None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:01.373176 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b23637-7817-4941-9c24-5184de84957c None admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:01.374240 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b23637-7817-4941-9c24-5184de84957c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:01.374577 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b23637-7817-4941-9c24-5184de84957c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:01.376727 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b23637-7817-4941-9c24-5184de84957c None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:01.379238 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:01.403075 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 374/759] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Jul 20 14:35:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:01.403075 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c2530-8194-44b5-8135-f8eb021c5abe None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:01.403075 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c2530-8194-44b5-8135-f8eb021c5abe None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:01.403075 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c2530-8194-44b5-8135-f8eb021c5abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:01.426286 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-6f5c2530-8194-44b5-8135-f8eb021c5abe None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:01.427714 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-6f5c2530-8194-44b5-8135-f8eb021c5abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:01.468596 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fbf984-9146-4010-ba6a-487c635c83d2 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:01.516190 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256be54-e9cf-4d79-89ae-6a8d7c41d5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:01.516921 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256be54-e9cf-4d79-89ae-6a8d7c41d5a5 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:01.517204 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256be54-e9cf-4d79-89ae-6a8d7c41d5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:01.517475 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256be54-e9cf-4d79-89ae-6a8d7c41d5a5 None admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:01.518265 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256be54-e9cf-4d79-89ae-6a8d7c41d5a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:01.518575 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256be54-e9cf-4d79-89ae-6a8d7c41d5a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:01.520056 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256be54-e9cf-4d79-89ae-6a8d7c41d5a5 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:01.535650 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 375/760] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:35:01.535650 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c2530-8194-44b5-8135-f8eb021c5abe None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:01.540682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 386/761] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:35:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:01.546171 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256be54-e9cf-4d79-89ae-6a8d7c41d5a5 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:01.651417 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:01.661879 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256be54-e9cf-4d79-89ae-6a8d7c41d5a5 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:01.679713 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c2530-8194-44b5-8135-f8eb021c5abe None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:01.702125 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:01.706737 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8901d68-e8a9-43d1-a7df-92a41f5f71d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:01.708156 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8901d68-e8a9-43d1-a7df-92a41f5f71d4 None admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:01.709087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8901d68-e8a9-43d1-a7df-92a41f5f71d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:01.710100 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8901d68-e8a9-43d1-a7df-92a41f5f71d4 None admin] PATH_INFO: `/v3/projects/59333e4f593346e184877d6bd9d22026/users/5d4a107ed04348b59d20d1703dbe04eb/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:01.713503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8901d68-e8a9-43d1-a7df-92a41f5f71d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=59333e4f593346e184877d6bd9d22026, user_id=5d4a107ed04348b59d20d1703dbe04eb, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:01.715763 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8901d68-e8a9-43d1-a7df-92a41f5f71d4 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:01.718067 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab1ccf-51d8-4f3f-bb38-5c7e8902ff37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:01.718901 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab1ccf-51d8-4f3f-bb38-5c7e8902ff37 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:01.719279 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab1ccf-51d8-4f3f-bb38-5c7e8902ff37 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:01.719624 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab1ccf-51d8-4f3f-bb38-5c7e8902ff37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:01.735331 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 387/762] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jul 20 14:35:01 2022] PUT /identity/v3/projects/59333e4f593346e184877d6bd9d22026/users/5d4a107ed04348b59d20d1703dbe04eb/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 14:35:01.736850 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8901d68-e8a9-43d1-a7df-92a41f5f71d4 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:01.810409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab1ccf-51d8-4f3f-bb38-5c7e8902ff37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:01.812364 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab1ccf-51d8-4f3f-bb38-5c7e8902ff37 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:01.933259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8901d68-e8a9-43d1-a7df-92a41f5f71d4 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:01.938247 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 376/763] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:35:01 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 14:35:01.945400 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab1ccf-51d8-4f3f-bb38-5c7e8902ff37 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:01.985948 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:01.998437 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f0e74-7fbe-48f0-b3bb-0b6d2ee9ad47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:01.999472 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f0e74-7fbe-48f0-b3bb-0b6d2ee9ad47 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:01.999684 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f0e74-7fbe-48f0-b3bb-0b6d2ee9ad47 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:01.999939 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f0e74-7fbe-48f0-b3bb-0b6d2ee9ad47 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:02.000606 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68f0e74-7fbe-48f0-b3bb-0b6d2ee9ad47 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:02.002095 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68f0e74-7fbe-48f0-b3bb-0b6d2ee9ad47 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:02.015537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-c68f0e74-7fbe-48f0-b3bb-0b6d2ee9ad47 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 14:35:02.015537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:02.015537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 14:35:02.015537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 14:35:02.015537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 14:35:02.015537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 14:35:02.015537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 14:35:02.015537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:02.015537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 14:35:02.015537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:02.015537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 14:35:02.015537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:02.015537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:02.015537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:02.015537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:02.015537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:02.016535 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:02.016535 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:02.016535 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:02.016535 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:02.016535 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:02.016535 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:02.016535 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:02.016535 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 14:35:02.016535 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 14:35:02.016535 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 14:35:02.016535 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 14:35:02.016535 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:02.016535 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:02.016535 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:02.016535 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:02.016535 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:02.017384 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:02.017384 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:02.017384 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:02.017384 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:02.017384 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:02.017384 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:02.017384 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:02.017384 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:02.017384 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:02.017384 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:02.017384 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:02.017384 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 14:35:02.017384 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:02.017384 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 14:35:02.017384 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:02.020770 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 388/764] 10.222.0.15 () {56 vars in 1129 bytes} [Wed Jul 20 14:35:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:02.023487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f0e74-7fbe-48f0-b3bb-0b6d2ee9ad47 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:02.046190 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:02.089236 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab1ccf-51d8-4f3f-bb38-5c7e8902ff37 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:02.127586 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436be5b3-59f7-4778-b868-eb149d6ca1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:02.128855 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436be5b3-59f7-4778-b868-eb149d6ca1d7 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:02.129390 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436be5b3-59f7-4778-b868-eb149d6ca1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:02.129894 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436be5b3-59f7-4778-b868-eb149d6ca1d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:02.135007 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f0e74-7fbe-48f0-b3bb-0b6d2ee9ad47 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:02.141529 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436be5b3-59f7-4778-b868-eb149d6ca1d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:02.143430 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436be5b3-59f7-4778-b868-eb149d6ca1d7 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:02.163252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-436be5b3-59f7-4778-b868-eb149d6ca1d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:02.163252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:02.163252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:02.163252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:02.163252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:02.163252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:02.163252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:02.163252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:02.163252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:02.163252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:02.163252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:02.163252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:02.163252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:35:02.163252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:35:02.163252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:35:02.164208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:35:02.164208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:02.164208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:02.164208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:02.164208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:02.164208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:02.164208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:02.164208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:02.164208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:02.164208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:02.164208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:02.164208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:02.164208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:02.164208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:02.164208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:02.164208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:02.164208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:02.165256 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:35:02.165256 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:35:02.165256 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:02.165256 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:02.168011 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 377/765] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:02.170266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6245d9f-be3c-4265-b617-81ca31e45405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:02.171231 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6245d9f-be3c-4265-b617-81ca31e45405 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:02.171720 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6245d9f-be3c-4265-b617-81ca31e45405 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:02.172216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6245d9f-be3c-4265-b617-81ca31e45405 None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:02.173554 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436be5b3-59f7-4778-b868-eb149d6ca1d7 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:02.174647 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6245d9f-be3c-4265-b617-81ca31e45405 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:02.174959 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6245d9f-be3c-4265-b617-81ca31e45405 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:02.176374 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6245d9f-be3c-4265-b617-81ca31e45405 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:02.191169 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 389/766] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Jul 20 14:35:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:02.201244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6245d9f-be3c-4265-b617-81ca31e45405 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:02.275356 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:02.308853 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436be5b3-59f7-4778-b868-eb149d6ca1d7 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:02.344433 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e781895-7ff4-4686-9c4d-34020ba32dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:02.345116 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e781895-7ff4-4686-9c4d-34020ba32dde None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:02.345398 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e781895-7ff4-4686-9c4d-34020ba32dde None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:02.345676 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e781895-7ff4-4686-9c4d-34020ba32dde None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:02.346414 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e781895-7ff4-4686-9c4d-34020ba32dde None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:02.346724 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e781895-7ff4-4686-9c4d-34020ba32dde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:02.348147 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e781895-7ff4-4686-9c4d-34020ba32dde None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:02.360957 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 378/767] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:02.366379 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e781895-7ff4-4686-9c4d-34020ba32dde None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:02.394698 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:02.423008 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6245d9f-be3c-4265-b617-81ca31e45405 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:02.461033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c50ac-205c-4961-a66d-56e46f6e0808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:02.462311 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c50ac-205c-4961-a66d-56e46f6e0808 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:02.462616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c50ac-205c-4961-a66d-56e46f6e0808 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:02.462901 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c50ac-205c-4961-a66d-56e46f6e0808 None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:02.463659 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c50ac-205c-4961-a66d-56e46f6e0808 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:02.463972 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c50ac-205c-4961-a66d-56e46f6e0808 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:02.465419 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c50ac-205c-4961-a66d-56e46f6e0808 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:02.471743 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 390/768] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Jul 20 14:35:02 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:02.485507 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c50ac-205c-4961-a66d-56e46f6e0808 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:02.563620 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e781895-7ff4-4686-9c4d-34020ba32dde None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:02.597738 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:02.599242 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29629ad-e8b2-4184-9f96-c79c8132cfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:02.600327 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29629ad-e8b2-4184-9f96-c79c8132cfc6 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:02.600622 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29629ad-e8b2-4184-9f96-c79c8132cfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:02.600928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29629ad-e8b2-4184-9f96-c79c8132cfc6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:02.608487 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29629ad-e8b2-4184-9f96-c79c8132cfc6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:02.609991 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29629ad-e8b2-4184-9f96-c79c8132cfc6 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:02.634263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-e29629ad-e8b2-4184-9f96-c79c8132cfc6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:35:02.634263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:02.634263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:02.634263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:02.634263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:02.634263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:02.634263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:02.634263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:02.634263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:02.634263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:02.634263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:02.634263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:02.634263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:35:02.634263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:35:02.634263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:35:02.635273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:35:02.635273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:02.635273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:02.635273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:02.635273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:02.635273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:02.635273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:02.635273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:02.635273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:02.635273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:02.635273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:02.635273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:02.635273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:02.635273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:02.635273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:02.635273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:02.635273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:02.636212 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:35:02.636212 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:35:02.636212 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:35:02.636212 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:35:02.636212 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:35:02.636212 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:35:02.636212 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:35:02.636212 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:02.636212 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 379/769] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:35:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:02.642697 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29629ad-e8b2-4184-9f96-c79c8132cfc6 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:02.680474 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c50ac-205c-4961-a66d-56e46f6e0808 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:02.752276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb31f41-5631-46ad-95eb-b357ffbbb5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:02.752873 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb31f41-5631-46ad-95eb-b357ffbbb5dc None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:02.753158 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb31f41-5631-46ad-95eb-b357ffbbb5dc None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:02.753434 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb31f41-5631-46ad-95eb-b357ffbbb5dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:02.754249 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb31f41-5631-46ad-95eb-b357ffbbb5dc None admin] RBAC: Adding query filter params (interface=public, service_id=c746e6e3e797486ab3434dcbcc01e092, region_id=RegionOne) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:02.754566 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb31f41-5631-46ad-95eb-b357ffbbb5dc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:02.756000 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb31f41-5631-46ad-95eb-b357ffbbb5dc None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:02.770811 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 391/770] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Jul 20 14:35:02 2022] GET /identity/v3/endpoints?service_id=c746e6e3e797486ab3434dcbcc01e092&interface=public®ion_id=RegionOne => generated 194 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:02.777267 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb31f41-5631-46ad-95eb-b357ffbbb5dc None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:02.799427 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:02.846815 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29629ad-e8b2-4184-9f96-c79c8132cfc6 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:02.891806 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c057d-d5dd-49a7-ad83-322e01439c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:02.892486 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c057d-d5dd-49a7-ad83-322e01439c40 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:02.892775 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c057d-d5dd-49a7-ad83-322e01439c40 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:02.893055 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c057d-d5dd-49a7-ad83-322e01439c40 None admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:02.893856 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7c057d-d5dd-49a7-ad83-322e01439c40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:02.894172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7c057d-d5dd-49a7-ad83-322e01439c40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:02.895781 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7c057d-d5dd-49a7-ad83-322e01439c40 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:02.909479 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 380/771] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:35:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:02.910844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca4785-8ab7-4a76-903b-d1e164fdf5fa None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:02.911165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca4785-8ab7-4a76-903b-d1e164fdf5fa None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:02.911452 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca4785-8ab7-4a76-903b-d1e164fdf5fa None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:02.917174 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 381/772] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:02.918178 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca4785-8ab7-4a76-903b-d1e164fdf5fa None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:02.940261 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb31f41-5631-46ad-95eb-b357ffbbb5dc None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:02.975954 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc2dcf-be2c-4ff1-a2de-4c27f6672f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:02.976979 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc2dcf-be2c-4ff1-a2de-4c27f6672f75 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:02.977444 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc2dcf-be2c-4ff1-a2de-4c27f6672f75 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:02.978068 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc2dcf-be2c-4ff1-a2de-4c27f6672f75 None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:02.978999 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc2dcf-be2c-4ff1-a2de-4c27f6672f75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:02.980594 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc2dcf-be2c-4ff1-a2de-4c27f6672f75 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:02.983477 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:02.987060 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 392/773] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Jul 20 14:35:02 2022] GET /identity/v3/services => generated 772 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:02.988634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0dbbb-baee-47ad-ad13-260fcf12e8b3 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:02.989280 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0dbbb-baee-47ad-ad13-260fcf12e8b3 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:02.989851 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0dbbb-baee-47ad-ad13-260fcf12e8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:03.008929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-dad0dbbb-baee-47ad-ad13-260fcf12e8b3 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:03.011123 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-dad0dbbb-baee-47ad-ad13-260fcf12e8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:03.093090 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca4785-8ab7-4a76-903b-d1e164fdf5fa None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:03.155396 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 393/774] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:35:03.160474 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87751f57-7b27-49f3-b746-b6ee3eeace8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:03.161410 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87751f57-7b27-49f3-b746-b6ee3eeace8f None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:03.161871 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87751f57-7b27-49f3-b746-b6ee3eeace8f None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:03.162321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87751f57-7b27-49f3-b746-b6ee3eeace8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:03.163423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87751f57-7b27-49f3-b746-b6ee3eeace8f None admin] RBAC: Adding query filter params (role.id=62e9fa78a7c44a16bb4540d3e8d7044d, scope.project.id=59333e4f593346e184877d6bd9d22026, user.id=2f8cf6d908314281a3ddc9d9513d39c2) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:03.163933 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87751f57-7b27-49f3-b746-b6ee3eeace8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:03.165575 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87751f57-7b27-49f3-b746-b6ee3eeace8f None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:03.172332 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 382/775] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:35:02 2022] GET /identity/v3/role_assignments?user.id=2f8cf6d908314281a3ddc9d9513d39c2&scope.project.id=59333e4f593346e184877d6bd9d22026&role.id=62e9fa78a7c44a16bb4540d3e8d7044d => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:03.185935 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0dbbb-baee-47ad-ad13-260fcf12e8b3 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:03.188659 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:03.284579 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6c384-d187-4270-924a-f1668ccff4e3 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:03.285154 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6c384-d187-4270-924a-f1668ccff4e3 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:03.285630 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6c384-d187-4270-924a-f1668ccff4e3 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:03.288622 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 383/776] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:03.293271 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c260ad-b24b-4974-8095-3ab68f723b87 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:03.293857 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c260ad-b24b-4974-8095-3ab68f723b87 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:03.294306 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c260ad-b24b-4974-8095-3ab68f723b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:03.315640 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0dbbb-baee-47ad-ad13-260fcf12e8b3 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:03.332418 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-f4c260ad-b24b-4974-8095-3ab68f723b87 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:03.333968 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-f4c260ad-b24b-4974-8095-3ab68f723b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:03.357670 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a4a4f-d202-496a-b9c4-2786f2594177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:03.358942 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a4a4f-d202-496a-b9c4-2786f2594177 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:03.359261 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a4a4f-d202-496a-b9c4-2786f2594177 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:03.359541 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a4a4f-d202-496a-b9c4-2786f2594177 None admin] PATH_INFO: `/v3/services/network` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:03.360214 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920a4a4f-d202-496a-b9c4-2786f2594177 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:03.361874 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920a4a4f-d202-496a-b9c4-2786f2594177 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:03.379169 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-920a4a4f-d202-496a-b9c4-2786f2594177 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 14:35:03.379169 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:03.379169 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 14:35:03.379169 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 14:35:03.379169 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 14:35:03.379169 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 14:35:03.379169 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 14:35:03.379169 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:03.379169 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 14:35:03.379169 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:03.379169 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 14:35:03.379169 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:03.379169 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:03.379169 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:03.379169 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:03.379169 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:03.379169 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:03.380597 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:03.380597 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:03.380597 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:03.380597 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:03.380597 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:03.380597 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:03.380597 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 14:35:03.380597 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 14:35:03.380597 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 14:35:03.380597 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 14:35:03.380597 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:03.380597 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:03.380597 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:03.380597 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:03.380597 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:03.380597 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:03.381751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:03.381751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:03.381751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:03.381751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:03.381751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:03.381751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:03.381751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:03.381751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:03.381751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:03.381751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:03.381751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 14:35:03.381751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:03.381751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 14:35:03.381751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:03.381751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 394/777] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:35:03 2022] GET /identity/v3/services/network => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:03.386190 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a4a4f-d202-496a-b9c4-2786f2594177 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:03.461082 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 384/778] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:35:03.487070 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c260ad-b24b-4974-8095-3ab68f723b87 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:03.521453 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:03.576072 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:03.586381 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a4a4f-d202-496a-b9c4-2786f2594177 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:03.655087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d8d09-98ea-4180-9d03-a7d6432cf061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:03.656343 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d8d09-98ea-4180-9d03-a7d6432cf061 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:03.656811 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d8d09-98ea-4180-9d03-a7d6432cf061 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:03.657091 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d8d09-98ea-4180-9d03-a7d6432cf061 None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:03.657806 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d8d09-98ea-4180-9d03-a7d6432cf061 None admin] RBAC: Adding query filter params (name=network) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:03.658110 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d8d09-98ea-4180-9d03-a7d6432cf061 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:03.659525 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d8d09-98ea-4180-9d03-a7d6432cf061 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:03.662190 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c260ad-b24b-4974-8095-3ab68f723b87 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:03.675711 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 395/779] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Jul 20 14:35:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:03.680548 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d8d09-98ea-4180-9d03-a7d6432cf061 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:03.740830 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4af554-5b5a-4692-a265-75daefb90e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:03.741650 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4af554-5b5a-4692-a265-75daefb90e2c None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:03.741935 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4af554-5b5a-4692-a265-75daefb90e2c None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:03.742212 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4af554-5b5a-4692-a265-75daefb90e2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:03.776029 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4af554-5b5a-4692-a265-75daefb90e2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:35:03.783734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-cd4af554-5b5a-4692-a265-75daefb90e2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:35:03.783734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:03.783734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:03.783734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:03.783734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:03.783734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:03.783734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:03.783734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:03.783734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:03.783734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:03.783734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:03.783734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:03.783734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:35:03.783734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:35:03.783734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:35:03.783734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:35:03.784644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:35:03.784644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:35:03.784644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:35:03.784644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:35:03.784644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:03.784644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:03.784644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:03.784644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:03.784644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:03.784644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:03.784644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:03.784644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:03.784644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:03.784644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:03.784644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:03.784644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:03.785508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:03.785508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:03.785508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:03.785508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:03.785508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:35:03.785508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:35:03.785508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:35:03.785508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:35:03.785508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:35:03.785508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:35:03.785508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:35:03.785508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:03.789303 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d8d09-98ea-4180-9d03-a7d6432cf061 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:03.791584 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 385/780] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:35:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:03.797107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4af554-5b5a-4692-a265-75daefb90e2c None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:03.823505 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:03.835152 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579de93c-43c4-4a52-9ea4-2cf57145dd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:03.835152 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579de93c-43c4-4a52-9ea4-2cf57145dd69 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:03.835152 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579de93c-43c4-4a52-9ea4-2cf57145dd69 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:03.835152 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579de93c-43c4-4a52-9ea4-2cf57145dd69 None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:03.835523 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579de93c-43c4-4a52-9ea4-2cf57145dd69 None admin] RBAC: Adding query filter params (type=network) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:03.835523 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579de93c-43c4-4a52-9ea4-2cf57145dd69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:03.840370 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579de93c-43c4-4a52-9ea4-2cf57145dd69 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:03.849481 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 396/781] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Jul 20 14:35:03 2022] GET /identity/v3/services?type=network => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:03.855532 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579de93c-43c4-4a52-9ea4-2cf57145dd69 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:04.003515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4af554-5b5a-4692-a265-75daefb90e2c None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:04.016126 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:04.035685 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579de93c-43c4-4a52-9ea4-2cf57145dd69 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:04.072253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bc55e-f7c5-4625-87d8-8e0091883d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:04.072944 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bc55e-f7c5-4625-87d8-8e0091883d4e None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:04.073227 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bc55e-f7c5-4625-87d8-8e0091883d4e None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:04.073505 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bc55e-f7c5-4625-87d8-8e0091883d4e None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:04.074801 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298bc55e-f7c5-4625-87d8-8e0091883d4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:04.075136 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298bc55e-f7c5-4625-87d8-8e0091883d4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:04.076540 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298bc55e-f7c5-4625-87d8-8e0091883d4e None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:04.080027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bbc034-ffeb-4f56-afcf-395f90a40263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:04.080609 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bbc034-ffeb-4f56-afcf-395f90a40263 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:04.080890 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bbc034-ffeb-4f56-afcf-395f90a40263 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:04.081167 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bbc034-ffeb-4f56-afcf-395f90a40263 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:04.082028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bbc034-ffeb-4f56-afcf-395f90a40263 None admin] RBAC: Adding query filter params (interface=public, service_id=e4b3f600147f49dfbaac24e4bee84904, region_id=RegionOne) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:04.082342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bbc034-ffeb-4f56-afcf-395f90a40263 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:04.083791 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bbc034-ffeb-4f56-afcf-395f90a40263 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:04.093830 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 386/783] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:35:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:04.094051 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 397/783] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Jul 20 14:35:03 2022] GET /identity/v3/endpoints?service_id=e4b3f600147f49dfbaac24e4bee84904&interface=public®ion_id=RegionOne => generated 194 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:04.100062 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bc55e-f7c5-4625-87d8-8e0091883d4e None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:04.102049 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bbc034-ffeb-4f56-afcf-395f90a40263 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:04.204398 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bc55e-f7c5-4625-87d8-8e0091883d4e None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:04.234454 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:04.240163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692311b4-ae94-465a-95ce-44f56726230b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:04.241052 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692311b4-ae94-465a-95ce-44f56726230b None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:04.241333 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692311b4-ae94-465a-95ce-44f56726230b None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:04.241613 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692311b4-ae94-465a-95ce-44f56726230b None admin] PATH_INFO: `/v3/services` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:04.242345 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692311b4-ae94-465a-95ce-44f56726230b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:04.243786 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692311b4-ae94-465a-95ce-44f56726230b None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:04.249522 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 387/784] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Jul 20 14:35:04 2022] GET /identity/v3/services => generated 772 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:04.250779 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512009bb-9018-4ec9-a49c-318bca86fa38 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:04.251083 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512009bb-9018-4ec9-a49c-318bca86fa38 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:04.251381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512009bb-9018-4ec9-a49c-318bca86fa38 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:04.255087 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 388/785] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:04.258859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680f6da-96ba-45c5-8ddd-81ee3139391c None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:04.259186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680f6da-96ba-45c5-8ddd-81ee3139391c None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:04.259467 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680f6da-96ba-45c5-8ddd-81ee3139391c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:04.282770 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-7680f6da-96ba-45c5-8ddd-81ee3139391c None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:04.284110 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-7680f6da-96ba-45c5-8ddd-81ee3139391c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:04.291910 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bbc034-ffeb-4f56-afcf-395f90a40263 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:04.313861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:04.336562 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ec4285-6167-4cb4-b6d3-c130ed93e354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:04.338266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec4285-6167-4cb4-b6d3-c130ed93e354 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:04.338753 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec4285-6167-4cb4-b6d3-c130ed93e354 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:04.339276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec4285-6167-4cb4-b6d3-c130ed93e354 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:04.352290 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ec4285-6167-4cb4-b6d3-c130ed93e354 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:04.354367 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ec4285-6167-4cb4-b6d3-c130ed93e354 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:04.359907 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 389/786] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:35:04.385101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-c0ec4285-6167-4cb4-b6d3-c130ed93e354 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:04.385101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:04.385101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:04.385101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:04.385101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:04.385101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:04.385101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:04.385101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:04.385101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:04.385101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:04.385101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:04.385101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:04.385101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:35:04.385101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:35:04.385101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:35:04.386084 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:35:04.386084 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:04.386084 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:04.386084 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:04.386084 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:04.386084 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:04.386084 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:04.386084 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:04.386084 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:04.386084 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:04.386084 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:04.386084 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:04.386084 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:04.386084 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:04.386084 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:04.386084 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:04.386084 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:04.386989 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:35:04.386989 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:35:04.386989 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:04.386989 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:04.386989 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 398/787] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:35:04.392201 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7680f6da-96ba-45c5-8ddd-81ee3139391c None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:04.404630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ec4285-6167-4cb4-b6d3-c130ed93e354 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:04.549754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7680f6da-96ba-45c5-8ddd-81ee3139391c None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:04.570145 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ec4285-6167-4cb4-b6d3-c130ed93e354 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:04.581262 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:04.590417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2534a-c0c4-4aa0-ad3a-2533cc5e0c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:04.591260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2534a-c0c4-4aa0-ad3a-2533cc5e0c6e None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:04.591549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2534a-c0c4-4aa0-ad3a-2533cc5e0c6e None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:04.591829 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2534a-c0c4-4aa0-ad3a-2533cc5e0c6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:04.604084 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff2534a-c0c4-4aa0-ad3a-2533cc5e0c6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:35:04.607707 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25511714-8f8b-4984-937a-a9a919a3e056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:04.608628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25511714-8f8b-4984-937a-a9a919a3e056 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:04.609114 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25511714-8f8b-4984-937a-a9a919a3e056 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:04.609591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25511714-8f8b-4984-937a-a9a919a3e056 None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:04.610527 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25511714-8f8b-4984-937a-a9a919a3e056 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:04.611076 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25511714-8f8b-4984-937a-a9a919a3e056 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:04.612682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25511714-8f8b-4984-937a-a9a919a3e056 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:04.612790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-cff2534a-c0c4-4aa0-ad3a-2533cc5e0c6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:35:04.612790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:04.612790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:04.612790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:04.612790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:04.612790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:04.612790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:04.612790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:04.612790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:04.612790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:04.612790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:04.612790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:04.612790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:35:04.612790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:35:04.612790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:35:04.612790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:35:04.613673 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:35:04.613673 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:35:04.613673 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:35:04.613673 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:35:04.613673 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:04.613673 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:04.613673 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:04.613673 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:04.613673 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:04.613673 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:04.613673 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:04.613673 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:04.613673 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:04.613673 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:04.613673 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:04.613673 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:04.614510 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:04.614510 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:04.614510 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:04.614510 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:04.614510 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:35:04.614510 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:35:04.614510 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:35:04.614510 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:35:04.614510 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:35:04.614510 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:35:04.614510 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:35:04.614510 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:04.614510 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 390/788] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:35:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:04.619965 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2534a-c0c4-4aa0-ad3a-2533cc5e0c6e None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:04.633998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 399/789] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:35:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:04.644149 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25511714-8f8b-4984-937a-a9a919a3e056 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:04.646427 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:04.728776 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2534a-c0c4-4aa0-ad3a-2533cc5e0c6e None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:04.765593 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f4fd9-bc7f-40cc-a678-116412c8b47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:04.766271 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f4fd9-bc7f-40cc-a678-116412c8b47f None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:04.766558 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f4fd9-bc7f-40cc-a678-116412c8b47f None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:04.766838 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f4fd9-bc7f-40cc-a678-116412c8b47f None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:04.767625 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417f4fd9-bc7f-40cc-a678-116412c8b47f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:04.767937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417f4fd9-bc7f-40cc-a678-116412c8b47f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:04.769342 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417f4fd9-bc7f-40cc-a678-116412c8b47f None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:04.785800 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 391/790] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:35:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:04.792093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f4fd9-bc7f-40cc-a678-116412c8b47f None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:04.860619 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25511714-8f8b-4984-937a-a9a919a3e056 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:04.927415 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:04.945804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f4fd9-bc7f-40cc-a678-116412c8b47f None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:04.947592 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45376cfe-554b-4c18-911e-922e1cd5dba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:04.948638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45376cfe-554b-4c18-911e-922e1cd5dba5 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:04.949125 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45376cfe-554b-4c18-911e-922e1cd5dba5 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:04.949601 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45376cfe-554b-4c18-911e-922e1cd5dba5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:04.964351 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45376cfe-554b-4c18-911e-922e1cd5dba5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:04.966120 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45376cfe-554b-4c18-911e-922e1cd5dba5 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:04.983838 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1553d0-ea95-4692-b74c-67e19df4aa7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:04.984646 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1553d0-ea95-4692-b74c-67e19df4aa7b None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:04.984941 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1553d0-ea95-4692-b74c-67e19df4aa7b None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:04.985215 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1553d0-ea95-4692-b74c-67e19df4aa7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:04.989514 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-45376cfe-554b-4c18-911e-922e1cd5dba5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 14:35:04.989514 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:04.989514 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:04.989514 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:04.989514 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:04.989514 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:04.989514 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:04.989514 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:04.989514 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:04.989514 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:04.989514 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:04.989514 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:04.989514 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:35:04.989514 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:35:04.989514 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:35:04.989514 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:35:04.990449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:04.990449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:04.990449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:35:04.990449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:35:04.990449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:35:04.990449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:35:04.990449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:04.990449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:04.990449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:04.990449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:04.990449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:04.990449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:04.990449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:04.990449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:04.990449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:04.990449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:04.990449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:04.990449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:04.991466 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:04.991466 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:04.991466 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:35:04.991466 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:35:04.991466 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:35:04.991466 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:35:04.991466 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:35:04.991466 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:35:04.991466 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 14:35:04.991466 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:04.994236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 400/791] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:35:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:04.998867 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1553d0-ea95-4692-b74c-67e19df4aa7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:04.998867 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1553d0-ea95-4692-b74c-67e19df4aa7b None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:05.000607 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45376cfe-554b-4c18-911e-922e1cd5dba5 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:05.032692 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-db1553d0-ea95-4692-b74c-67e19df4aa7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:05.032692 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:05.032692 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:05.032692 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:05.032692 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:05.032692 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:05.032692 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:05.032692 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:05.032692 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:05.032692 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:05.032692 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:05.032692 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:05.032692 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:35:05.032692 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:35:05.032692 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:35:05.033995 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:35:05.033995 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:05.033995 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:05.033995 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:05.033995 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:05.033995 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:05.033995 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:05.033995 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:05.033995 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:05.033995 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:05.033995 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:05.033995 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:05.033995 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:05.033995 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:05.033995 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:05.033995 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:05.033995 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:05.034900 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:35:05.034900 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:35:05.034900 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:05.034900 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:05.045107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 392/792] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:05.051105 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1553d0-ea95-4692-b74c-67e19df4aa7b None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:05.068306 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:05.196252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1553d0-ea95-4692-b74c-67e19df4aa7b None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:05.248592 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab83dd6-8c67-45b2-ab98-1137e288f75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:05.249692 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab83dd6-8c67-45b2-ab98-1137e288f75d None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:05.250118 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab83dd6-8c67-45b2-ab98-1137e288f75d None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:05.250600 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab83dd6-8c67-45b2-ab98-1137e288f75d None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:05.251660 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab83dd6-8c67-45b2-ab98-1137e288f75d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:05.251979 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab83dd6-8c67-45b2-ab98-1137e288f75d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:05.253395 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab83dd6-8c67-45b2-ab98-1137e288f75d None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:05.256756 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:05.261994 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45376cfe-554b-4c18-911e-922e1cd5dba5 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:05.270158 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 393/793] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:35:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:05.280419 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab83dd6-8c67-45b2-ab98-1137e288f75d None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:05.340572 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ddf73b-ac9b-44b8-8c2f-3ff11b81c05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:05.341285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ddf73b-ac9b-44b8-8c2f-3ff11b81c05b None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:05.341541 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ddf73b-ac9b-44b8-8c2f-3ff11b81c05b None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:05.341817 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ddf73b-ac9b-44b8-8c2f-3ff11b81c05b None admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:05.342674 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ddf73b-ac9b-44b8-8c2f-3ff11b81c05b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:05.342987 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ddf73b-ac9b-44b8-8c2f-3ff11b81c05b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:05.344466 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ddf73b-ac9b-44b8-8c2f-3ff11b81c05b None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:05.383197 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 401/794] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Jul 20 14:35:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:05.392477 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ddf73b-ac9b-44b8-8c2f-3ff11b81c05b None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:05.439421 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab83dd6-8c67-45b2-ab98-1137e288f75d None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:05.498691 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:05.513401 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ddf73b-ac9b-44b8-8c2f-3ff11b81c05b None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:05.555025 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97df8f7-3f55-4aa6-95e3-e53169caccc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:05.556341 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97df8f7-3f55-4aa6-95e3-e53169caccc6 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:05.556813 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97df8f7-3f55-4aa6-95e3-e53169caccc6 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:05.557241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97df8f7-3f55-4aa6-95e3-e53169caccc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:05.569576 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97df8f7-3f55-4aa6-95e3-e53169caccc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:05.571468 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97df8f7-3f55-4aa6-95e3-e53169caccc6 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:05.594174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-b97df8f7-3f55-4aa6-95e3-e53169caccc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:05.594174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:05.594174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:05.594174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:05.594174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:05.594174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:05.594174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:05.594174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:05.594174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:05.594174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:05.594174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:05.594174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:05.594174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:35:05.594174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:35:05.594174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:35:05.595409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:35:05.595409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:05.595409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:05.595409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:05.595409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:05.595409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:05.595409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:05.595409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:05.595409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:05.595409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:05.595409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:05.595409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:05.595409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:05.595409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:05.595409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:05.595409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:05.595409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:05.596571 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:35:05.596571 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:35:05.596571 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:05.596571 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:05.603992 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 402/795] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:05.604586 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9f30f-5bbc-40fe-a4fc-76b7839b43d7 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:05.604853 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9f30f-5bbc-40fe-a4fc-76b7839b43d7 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:05.605131 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9f30f-5bbc-40fe-a4fc-76b7839b43d7 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:05.606702 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 403/796] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:05.611469 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9f30f-5bbc-40fe-a4fc-76b7839b43d7 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:05.621321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31e2ea-4e50-41e9-9ab2-57b544cd1d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:05.622454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31e2ea-4e50-41e9-9ab2-57b544cd1d13 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:05.623321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31e2ea-4e50-41e9-9ab2-57b544cd1d13 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:05.623824 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31e2ea-4e50-41e9-9ab2-57b544cd1d13 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:05.633972 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31e2ea-4e50-41e9-9ab2-57b544cd1d13 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:05.636464 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31e2ea-4e50-41e9-9ab2-57b544cd1d13 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:05.656165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-6a31e2ea-4e50-41e9-9ab2-57b544cd1d13 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 14:35:05.656165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:05.656165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:05.656165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:05.656165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:05.656165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:05.656165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:05.656165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:05.656165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:05.656165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:05.656165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:05.656165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:05.656165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:35:05.656165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:35:05.656165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:35:05.656165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:35:05.657399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:05.657399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:05.657399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:35:05.657399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:35:05.657399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:35:05.657399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:35:05.657399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:05.657399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:05.657399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:05.657399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:05.657399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:05.657399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:05.657399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:05.657399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:05.657399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:05.657399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:05.657399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:05.657399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:05.658596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:05.658596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:05.658596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:35:05.658596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:35:05.658596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:35:05.658596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:35:05.658596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:35:05.658596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:35:05.658596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 14:35:05.658596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:05.660511 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 394/797] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Jul 20 14:35:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:05.666289 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eaa44a-45c6-416c-bb59-4449480b9018 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:05.666576 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eaa44a-45c6-416c-bb59-4449480b9018 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:05.666852 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eaa44a-45c6-416c-bb59-4449480b9018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:05.685772 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-04eaa44a-45c6-416c-bb59-4449480b9018 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:05.686961 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-04eaa44a-45c6-416c-bb59-4449480b9018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:05.724386 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9f30f-5bbc-40fe-a4fc-76b7839b43d7 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:05.739133 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:05.756562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 395/798] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:35:05.757906 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b455b86-3bbc-4829-929a-7535fd739e87 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:05.758193 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b455b86-3bbc-4829-929a-7535fd739e87 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:05.758470 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b455b86-3bbc-4829-929a-7535fd739e87 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:05.763080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 396/799] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:05.763080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b455b86-3bbc-4829-929a-7535fd739e87 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:05.773143 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e5766-d3a4-4b6c-ac89-caaf9b725e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:05.773907 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e5766-d3a4-4b6c-ac89-caaf9b725e69 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:05.774268 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e5766-d3a4-4b6c-ac89-caaf9b725e69 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:05.774623 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e5766-d3a4-4b6c-ac89-caaf9b725e69 None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:05.775563 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498e5766-d3a4-4b6c-ac89-caaf9b725e69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:05.775960 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498e5766-d3a4-4b6c-ac89-caaf9b725e69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:05.777550 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498e5766-d3a4-4b6c-ac89-caaf9b725e69 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:05.786710 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:05.795109 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 404/800] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:35:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:05.795109 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd2e9c-ff67-4ef2-a5b4-ae31674efbb0 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:05.795109 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd2e9c-ff67-4ef2-a5b4-ae31674efbb0 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:05.795456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd2e9c-ff67-4ef2-a5b4-ae31674efbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:05.813438 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-96fd2e9c-ff67-4ef2-a5b4-ae31674efbb0 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:05.814648 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-96fd2e9c-ff67-4ef2-a5b4-ae31674efbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:05.872687 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b455b86-3bbc-4829-929a-7535fd739e87 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:05.887388 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 405/801] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:35:05.887388 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd2e9c-ff67-4ef2-a5b4-ae31674efbb0 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:05.914482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf8307-b6ad-4bbd-8953-a5e8610b197c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:05.915401 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf8307-b6ad-4bbd-8953-a5e8610b197c None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:05.915688 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf8307-b6ad-4bbd-8953-a5e8610b197c None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:05.915966 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf8307-b6ad-4bbd-8953-a5e8610b197c None admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:05.916817 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf8307-b6ad-4bbd-8953-a5e8610b197c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:05.917131 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf8307-b6ad-4bbd-8953-a5e8610b197c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:05.918602 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf8307-b6ad-4bbd-8953-a5e8610b197c None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:05.937084 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 397/802] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Jul 20 14:35:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:05.937955 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf8307-b6ad-4bbd-8953-a5e8610b197c None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:05.985436 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:06.003526 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd2e9c-ff67-4ef2-a5b4-ae31674efbb0 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:06.037545 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:06.040804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf8307-b6ad-4bbd-8953-a5e8610b197c None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:06.045688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a038d-e155-418f-8bf6-8ac3f2dc0875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:06.046518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a038d-e155-418f-8bf6-8ac3f2dc0875 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:06.046809 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a038d-e155-418f-8bf6-8ac3f2dc0875 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:06.047116 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a038d-e155-418f-8bf6-8ac3f2dc0875 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:06.048507 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947a038d-e155-418f-8bf6-8ac3f2dc0875 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:06.050250 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947a038d-e155-418f-8bf6-8ac3f2dc0875 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:06.063254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-947a038d-e155-418f-8bf6-8ac3f2dc0875 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 14:35:06.063254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:06.063254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 14:35:06.063254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 14:35:06.063254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 14:35:06.063254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 14:35:06.063254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 14:35:06.063254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:06.063254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 14:35:06.063254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:06.063254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 14:35:06.063254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:06.063254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:06.063254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:06.063254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:06.063254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:06.064241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:06.064241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:06.064241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:06.064241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:06.064241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:06.064241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:06.064241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:06.064241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 14:35:06.064241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 14:35:06.064241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 14:35:06.064241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 14:35:06.064241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:06.064241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:06.064241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:06.064241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:06.064241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:06.065104 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:06.065104 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:06.065104 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:06.065104 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:06.065104 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:06.065104 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:06.065104 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:06.065104 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:06.065104 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:06.065104 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:06.065104 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:06.065104 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 14:35:06.065104 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:06.065104 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 14:35:06.065104 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:06.065104 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 406/803] 10.222.0.15 () {56 vars in 1129 bytes} [Wed Jul 20 14:35:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:06.065961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a038d-e155-418f-8bf6-8ac3f2dc0875 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:06.083771 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a632b1-649d-4dbc-ac2b-ce41ea39f956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:06.084697 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a632b1-649d-4dbc-ac2b-ce41ea39f956 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:06.084983 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a632b1-649d-4dbc-ac2b-ce41ea39f956 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:06.085336 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a632b1-649d-4dbc-ac2b-ce41ea39f956 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:06.092979 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a632b1-649d-4dbc-ac2b-ce41ea39f956 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:06.094866 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a632b1-649d-4dbc-ac2b-ce41ea39f956 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:06.109650 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-f2a632b1-649d-4dbc-ac2b-ce41ea39f956 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:35:06.109650 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:06.109650 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:06.109650 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:06.109650 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:06.109650 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:06.109650 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:06.109650 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:06.109650 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:06.109650 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:06.109650 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:06.109650 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:06.109650 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:35:06.109650 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:35:06.109650 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:35:06.110588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:35:06.110588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:06.110588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:06.110588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:06.110588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:06.110588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:06.110588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:06.110588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:06.110588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:06.110588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:06.110588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:06.110588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:06.110588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:06.110588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:06.110588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:06.110588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:06.110588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:06.111537 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:35:06.111537 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:35:06.111537 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:35:06.111537 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:35:06.111537 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:35:06.111537 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:35:06.111537 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:35:06.111537 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:06.115737 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 398/804] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:35:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:06.116603 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a632b1-649d-4dbc-ac2b-ce41ea39f956 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:06.174297 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a038d-e155-418f-8bf6-8ac3f2dc0875 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:06.210750 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe46ec0-3c82-4275-89e5-6db7b34141ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:06.211621 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe46ec0-3c82-4275-89e5-6db7b34141ca None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:06.211911 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe46ec0-3c82-4275-89e5-6db7b34141ca None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:06.212192 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe46ec0-3c82-4275-89e5-6db7b34141ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:06.222873 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe46ec0-3c82-4275-89e5-6db7b34141ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:35:06.224097 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a632b1-649d-4dbc-ac2b-ce41ea39f956 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:06.226505 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:06.230914 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-9fe46ec0-3c82-4275-89e5-6db7b34141ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:35:06.230914 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:06.230914 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:06.230914 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:06.230914 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:06.230914 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:06.230914 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:06.230914 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:06.230914 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:06.230914 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:06.230914 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:06.230914 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:06.230914 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:35:06.230914 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:35:06.230914 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:35:06.230914 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:35:06.231910 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:35:06.231910 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:35:06.231910 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:35:06.231910 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:35:06.231910 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:06.231910 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:06.231910 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:06.231910 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:06.231910 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:06.231910 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:06.231910 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:06.231910 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:06.231910 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:06.231910 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:06.231910 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:06.231910 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:06.237608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:06.237608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:06.237608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:06.237608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:06.237608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:35:06.237608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:35:06.237608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:35:06.237608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:35:06.237608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:35:06.237608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:35:06.237608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:35:06.237608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:06.237608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 407/805] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jul 20 14:35:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:06.237608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe46ec0-3c82-4275-89e5-6db7b34141ca None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:06.268311 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ef320-1bb8-4030-ab73-b14c1133e31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:06.269241 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ef320-1bb8-4030-ab73-b14c1133e31e None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:06.269527 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ef320-1bb8-4030-ab73-b14c1133e31e None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:06.269813 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ef320-1bb8-4030-ab73-b14c1133e31e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:06.295504 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:06.343470 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe46ec0-3c82-4275-89e5-6db7b34141ca None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:06.344106 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1ef320-1bb8-4030-ab73-b14c1133e31e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:06.345125 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1ef320-1bb8-4030-ab73-b14c1133e31e None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:06.378979 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8ab85-1271-4904-8d5f-b491aee14b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:06.379982 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8ab85-1271-4904-8d5f-b491aee14b35 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:06.380462 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8ab85-1271-4904-8d5f-b491aee14b35 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:06.380749 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8ab85-1271-4904-8d5f-b491aee14b35 None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:06.381523 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f8ab85-1271-4904-8d5f-b491aee14b35 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:06.381838 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f8ab85-1271-4904-8d5f-b491aee14b35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:06.383240 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f8ab85-1271-4904-8d5f-b491aee14b35 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:06.388172 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 408/806] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Jul 20 14:35:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:06.388792 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8ab85-1271-4904-8d5f-b491aee14b35 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:06.460257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 399/807] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:35:06 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 14:35:06.460257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ef320-1bb8-4030-ab73-b14c1133e31e None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:06.460822 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:06.506589 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8ab85-1271-4904-8d5f-b491aee14b35 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:06.525225 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:06.549170 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b94e2-fe82-4779-82ad-870a7b5f946e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:06.550300 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b94e2-fe82-4779-82ad-870a7b5f946e None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:06.550600 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b94e2-fe82-4779-82ad-870a7b5f946e None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:06.550885 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b94e2-fe82-4779-82ad-870a7b5f946e None admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:06.552157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b94e2-fe82-4779-82ad-870a7b5f946e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:06.552718 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b94e2-fe82-4779-82ad-870a7b5f946e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:06.554593 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b94e2-fe82-4779-82ad-870a7b5f946e None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:06.569769 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ef320-1bb8-4030-ab73-b14c1133e31e None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:06.574955 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 409/808] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:35:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:06.575766 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b94e2-fe82-4779-82ad-870a7b5f946e None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:06.607231 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf8e68-22ce-4feb-96b0-75b5ada7a1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:06.607904 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf8e68-22ce-4feb-96b0-75b5ada7a1c7 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:06.608192 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf8e68-22ce-4feb-96b0-75b5ada7a1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:06.608469 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf8e68-22ce-4feb-96b0-75b5ada7a1c7 None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:06.609209 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cf8e68-22ce-4feb-96b0-75b5ada7a1c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:06.609517 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cf8e68-22ce-4feb-96b0-75b5ada7a1c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:06.610872 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cf8e68-22ce-4feb-96b0-75b5ada7a1c7 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:06.623619 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 400/809] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jul 20 14:35:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:06.623619 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf8e68-22ce-4feb-96b0-75b5ada7a1c7 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:06.686934 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b94e2-fe82-4779-82ad-870a7b5f946e None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:06.689385 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:06.721579 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897d56d-f248-4301-8cf7-eb35b990c0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:06.722260 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897d56d-f248-4301-8cf7-eb35b990c0dc None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:06.722542 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897d56d-f248-4301-8cf7-eb35b990c0dc None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:06.722821 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897d56d-f248-4301-8cf7-eb35b990c0dc None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:06.723571 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897d56d-f248-4301-8cf7-eb35b990c0dc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:06.723876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897d56d-f248-4301-8cf7-eb35b990c0dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:06.725314 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897d56d-f248-4301-8cf7-eb35b990c0dc None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:06.725796 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf8e68-22ce-4feb-96b0-75b5ada7a1c7 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:06.730566 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 410/810] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Jul 20 14:35:06 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:06.731367 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897d56d-f248-4301-8cf7-eb35b990c0dc None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:06.760218 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b996f22-35a2-4ffa-be58-882a99990e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:06.761023 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b996f22-35a2-4ffa-be58-882a99990e37 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:06.761305 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b996f22-35a2-4ffa-be58-882a99990e37 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:06.761682 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b996f22-35a2-4ffa-be58-882a99990e37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:06.772488 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b996f22-35a2-4ffa-be58-882a99990e37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:06.773955 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b996f22-35a2-4ffa-be58-882a99990e37 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:06.777211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:06.790822 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-7b996f22-35a2-4ffa-be58-882a99990e37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:06.790822 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:06.790822 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:06.790822 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:06.790822 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:06.790822 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:06.790822 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:06.790822 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:06.790822 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:06.790822 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:06.790822 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:06.790822 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:06.790822 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:35:06.790822 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:35:06.790822 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:35:06.791859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:35:06.791859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:06.791859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:06.791859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:06.791859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:06.791859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:06.791859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:06.791859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:06.791859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:06.791859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:06.791859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:06.791859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:06.791859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:06.791859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:06.791859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:06.791859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:06.791859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:06.792754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:35:06.792754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:35:06.792754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:06.792754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:06.792754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 401/811] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:06.793041 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b996f22-35a2-4ffa-be58-882a99990e37 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:06.837480 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897d56d-f248-4301-8cf7-eb35b990c0dc None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:06.873926 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec479e4-781f-4480-84b1-30c78318e7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:06.874823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec479e4-781f-4480-84b1-30c78318e7af None admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:06.875131 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec479e4-781f-4480-84b1-30c78318e7af None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:06.875399 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec479e4-781f-4480-84b1-30c78318e7af None admin] PATH_INFO: `/v3/projects/59333e4f593346e184877d6bd9d22026/users/37f4844988474d7a91133d77906611a5/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:06.877391 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec479e4-781f-4480-84b1-30c78318e7af None admin] RBAC: Authorizing `identity:create_grant(project_id=59333e4f593346e184877d6bd9d22026, user_id=37f4844988474d7a91133d77906611a5, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:06.878822 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec479e4-781f-4480-84b1-30c78318e7af None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:06.892121 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 411/812] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jul 20 14:35:06 2022] PUT /identity/v3/projects/59333e4f593346e184877d6bd9d22026/users/37f4844988474d7a91133d77906611a5/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:35:06.893394 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b9791-a46f-43bf-8963-2db998a1b5fe None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:06.893681 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b9791-a46f-43bf-8963-2db998a1b5fe None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:06.893960 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b9791-a46f-43bf-8963-2db998a1b5fe None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:06.895252 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 412/813] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:06.895941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b9791-a46f-43bf-8963-2db998a1b5fe None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:06.910069 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b996f22-35a2-4ffa-be58-882a99990e37 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:06.930432 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:06.950951 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db924bb-49a0-41dc-86cf-6f45f6cdc426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:06.952080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db924bb-49a0-41dc-86cf-6f45f6cdc426 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:06.952519 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db924bb-49a0-41dc-86cf-6f45f6cdc426 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:06.952798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db924bb-49a0-41dc-86cf-6f45f6cdc426 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:06.966172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db924bb-49a0-41dc-86cf-6f45f6cdc426 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:06.967790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db924bb-49a0-41dc-86cf-6f45f6cdc426 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:06.987460 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-2db924bb-49a0-41dc-86cf-6f45f6cdc426 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:06.987460 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:06.987460 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:06.987460 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:06.987460 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:06.987460 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:06.987460 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:06.987460 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:06.987460 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:06.987460 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:06.987460 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:06.987460 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:06.987460 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:35:06.987460 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:35:06.987460 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:35:06.988509 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:35:06.988509 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:06.988509 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:06.988509 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:06.988509 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:06.988509 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:06.988509 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:06.988509 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:06.988509 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:06.988509 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:06.988509 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:06.988509 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:06.988509 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:06.988509 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:06.988509 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:06.988509 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:06.988509 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:06.994403 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:35:06.994403 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:35:06.994403 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:06.994403 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:06.994403 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 402/814] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:06.994403 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db924bb-49a0-41dc-86cf-6f45f6cdc426 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:07.023643 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b9791-a46f-43bf-8963-2db998a1b5fe None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:07.025554 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:07.068021 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8291f83-c1b9-44f8-91d9-7b9d61b5dd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:07.068737 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8291f83-c1b9-44f8-91d9-7b9d61b5dd42 None admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:07.069022 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8291f83-c1b9-44f8-91d9-7b9d61b5dd42 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:07.069299 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8291f83-c1b9-44f8-91d9-7b9d61b5dd42 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:07.070054 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8291f83-c1b9-44f8-91d9-7b9d61b5dd42 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:07.071641 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8291f83-c1b9-44f8-91d9-7b9d61b5dd42 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:07.092942 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 413/815] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Jul 20 14:35:06 2022] POST /identity/v3/endpoints => generated 346 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:35:07.095097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c895b-fc8e-44a4-8aa2-31832b7f115a None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:07.095097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c895b-fc8e-44a4-8aa2-31832b7f115a None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:07.095097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c895b-fc8e-44a4-8aa2-31832b7f115a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:07.114370 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-6f7c895b-fc8e-44a4-8aa2-31832b7f115a None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:07.115718 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-6f7c895b-fc8e-44a4-8aa2-31832b7f115a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:07.123353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db924bb-49a0-41dc-86cf-6f45f6cdc426 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:07.187682 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87578c32-5746-46f0-883f-3b26bb761c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:07.188911 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87578c32-5746-46f0-883f-3b26bb761c16 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:07.189400 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87578c32-5746-46f0-883f-3b26bb761c16 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:07.190261 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87578c32-5746-46f0-883f-3b26bb761c16 None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:07.191468 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87578c32-5746-46f0-883f-3b26bb761c16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:07.191933 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87578c32-5746-46f0-883f-3b26bb761c16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:07.194509 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87578c32-5746-46f0-883f-3b26bb761c16 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:07.198343 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 414/816] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:35:07.199530 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c895b-fc8e-44a4-8aa2-31832b7f115a None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:07.211495 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 403/817] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:07.220120 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87578c32-5746-46f0-883f-3b26bb761c16 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:07.255647 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:07.321487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:07.336051 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c895b-fc8e-44a4-8aa2-31832b7f115a None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:07.358917 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87578c32-5746-46f0-883f-3b26bb761c16 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:07.377617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3d935-0efc-4919-ac6c-139c58ab289e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:07.378290 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3d935-0efc-4919-ac6c-139c58ab289e None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:07.378573 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3d935-0efc-4919-ac6c-139c58ab289e None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:07.378853 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3d935-0efc-4919-ac6c-139c58ab289e None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:07.379726 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca3d935-0efc-4919-ac6c-139c58ab289e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:07.380039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca3d935-0efc-4919-ac6c-139c58ab289e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:07.381476 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca3d935-0efc-4919-ac6c-139c58ab289e None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:07.393878 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 415/818] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:07.394710 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3d935-0efc-4919-ac6c-139c58ab289e None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:07.394834 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda30ec-bb40-43dd-8bfa-5fae6bceb9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:07.395721 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda30ec-bb40-43dd-8bfa-5fae6bceb9c0 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:07.396008 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda30ec-bb40-43dd-8bfa-5fae6bceb9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:07.396283 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda30ec-bb40-43dd-8bfa-5fae6bceb9c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:07.408248 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda30ec-bb40-43dd-8bfa-5fae6bceb9c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:07.410195 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda30ec-bb40-43dd-8bfa-5fae6bceb9c0 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:07.430953 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-5cda30ec-bb40-43dd-8bfa-5fae6bceb9c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:35:07.430953 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:07.430953 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:07.430953 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:07.430953 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:07.430953 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:07.430953 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:07.430953 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:07.430953 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:07.430953 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:07.430953 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:07.430953 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:07.430953 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:35:07.430953 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:35:07.430953 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:35:07.432005 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:35:07.432005 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:07.432005 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:07.432005 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:07.432005 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:07.432005 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:07.432005 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:07.432005 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:07.432005 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:07.432005 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:07.432005 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:07.432005 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:07.432005 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:07.432005 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:07.432005 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:07.432005 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:07.432005 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:07.432910 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:35:07.432910 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:35:07.432910 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:35:07.432910 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:35:07.432910 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:35:07.432910 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:35:07.432910 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:35:07.432910 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:07.435802 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 404/819] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:35:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:07.437488 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda30ec-bb40-43dd-8bfa-5fae6bceb9c0 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:07.517387 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3d935-0efc-4919-ac6c-139c58ab289e None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:07.553873 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16218b0f-09f1-412b-b4eb-4ae5243b564a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:07.554871 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16218b0f-09f1-412b-b4eb-4ae5243b564a None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:07.555284 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16218b0f-09f1-412b-b4eb-4ae5243b564a None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:07.555775 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16218b0f-09f1-412b-b4eb-4ae5243b564a None admin] PATH_INFO: `/v3/services/network` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:07.556978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16218b0f-09f1-412b-b4eb-4ae5243b564a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:07.559443 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16218b0f-09f1-412b-b4eb-4ae5243b564a None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:07.565575 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:07.575039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-16218b0f-09f1-412b-b4eb-4ae5243b564a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 14:35:07.575039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:07.575039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 14:35:07.575039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 14:35:07.575039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 14:35:07.575039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 14:35:07.575039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 14:35:07.575039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:07.575039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 14:35:07.575039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:07.575039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 14:35:07.575039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:07.575039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:07.575039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:07.575039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:07.575039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:07.575039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:07.576296 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:07.576296 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:07.576296 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:07.576296 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:07.576296 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:07.576296 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:07.576296 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 14:35:07.576296 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 14:35:07.576296 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 14:35:07.576296 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 14:35:07.576296 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:07.576296 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:07.576296 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:07.576296 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:07.576296 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:07.576296 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:07.577459 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:07.577459 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:07.577459 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:07.577459 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:07.577459 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:07.577459 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:07.577459 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:07.577459 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:07.577459 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:07.577459 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:07.577459 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 14:35:07.577459 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:07.577459 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 14:35:07.577459 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:07.579247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 416/820] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:35:07 2022] GET /identity/v3/services/network => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:07.580269 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16218b0f-09f1-412b-b4eb-4ae5243b564a None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:07.587416 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda30ec-bb40-43dd-8bfa-5fae6bceb9c0 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:07.598061 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:07.634657 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01abfa2b-a88b-4731-bdc1-3b634e93a3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:07.635188 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01abfa2b-a88b-4731-bdc1-3b634e93a3d5 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:07.635475 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01abfa2b-a88b-4731-bdc1-3b634e93a3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:07.635750 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01abfa2b-a88b-4731-bdc1-3b634e93a3d5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:07.644054 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01abfa2b-a88b-4731-bdc1-3b634e93a3d5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:07.645515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01abfa2b-a88b-4731-bdc1-3b634e93a3d5 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:07.661661 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-01abfa2b-a88b-4731-bdc1-3b634e93a3d5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 14:35:07.661661 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:07.661661 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:07.661661 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:07.661661 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:07.661661 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:07.661661 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:07.661661 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:07.661661 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:07.661661 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:07.661661 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:07.661661 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:07.661661 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:35:07.661661 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:35:07.661661 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:35:07.661661 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:35:07.662734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:07.662734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:07.662734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:35:07.662734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:35:07.662734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:35:07.662734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:35:07.662734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:07.662734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:07.662734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:07.662734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:07.662734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:07.662734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:07.662734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:07.662734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:07.662734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:07.662734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:07.662734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:07.662734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:07.663732 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:07.663732 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:07.663732 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:35:07.663732 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:35:07.663732 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:35:07.663732 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:35:07.663732 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:35:07.663732 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:35:07.663732 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 14:35:07.663732 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:07.667419 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 405/821] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Jul 20 14:35:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:07.668413 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01abfa2b-a88b-4731-bdc1-3b634e93a3d5 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:07.708311 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16218b0f-09f1-412b-b4eb-4ae5243b564a None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:07.747906 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2976b-cd4f-4571-a069-56a089776379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:07.748613 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2976b-cd4f-4571-a069-56a089776379 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:07.749206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2976b-cd4f-4571-a069-56a089776379 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:07.749351 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2976b-cd4f-4571-a069-56a089776379 None admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:07.750069 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a2976b-cd4f-4571-a069-56a089776379 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:07.750378 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a2976b-cd4f-4571-a069-56a089776379 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:07.751989 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a2976b-cd4f-4571-a069-56a089776379 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:07.770569 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 417/822] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:35:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:07.770569 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2976b-cd4f-4571-a069-56a089776379 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:07.807942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01abfa2b-a88b-4731-bdc1-3b634e93a3d5 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:07.843067 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a6ea8-55b5-45cf-9d1b-144d65b7e98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:07.843765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a6ea8-55b5-45cf-9d1b-144d65b7e98b None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:07.844420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a6ea8-55b5-45cf-9d1b-144d65b7e98b None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:07.844420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a6ea8-55b5-45cf-9d1b-144d65b7e98b None admin] PATH_INFO: `/v3/services` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:07.845072 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a6ea8-55b5-45cf-9d1b-144d65b7e98b None admin] RBAC: Adding query filter params (name=network) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:07.845384 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a6ea8-55b5-45cf-9d1b-144d65b7e98b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:07.846765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a6ea8-55b5-45cf-9d1b-144d65b7e98b None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:07.849148 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:07.855778 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 406/823] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Jul 20 14:35:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:07.857028 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a6ea8-55b5-45cf-9d1b-144d65b7e98b None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:07.872841 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:07.887429 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2976b-cd4f-4571-a069-56a089776379 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:07.923798 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6f24e-71e4-4ae9-a7cd-96fda596dccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:07.924506 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6f24e-71e4-4ae9-a7cd-96fda596dccb None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:07.924796 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6f24e-71e4-4ae9-a7cd-96fda596dccb None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:07.925076 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6f24e-71e4-4ae9-a7cd-96fda596dccb None admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:07.925945 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6f24e-71e4-4ae9-a7cd-96fda596dccb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:07.926403 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6f24e-71e4-4ae9-a7cd-96fda596dccb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:07.927883 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6f24e-71e4-4ae9-a7cd-96fda596dccb None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:07.951318 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 418/824] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Jul 20 14:35:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:07.952686 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6f24e-71e4-4ae9-a7cd-96fda596dccb None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:07.994807 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a6ea8-55b5-45cf-9d1b-144d65b7e98b None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:08.030546 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa07183-7720-47c2-8872-b43b59e8886d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:08.031312 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa07183-7720-47c2-8872-b43b59e8886d None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:08.031603 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa07183-7720-47c2-8872-b43b59e8886d None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:08.031885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa07183-7720-47c2-8872-b43b59e8886d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:08.033213 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa07183-7720-47c2-8872-b43b59e8886d None admin] RBAC: Adding query filter params (role.id=49bd5abf255048a0abaa9538e309f516, scope.project.id=59333e4f593346e184877d6bd9d22026, user.id=5d4a107ed04348b59d20d1703dbe04eb) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:08.033213 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa07183-7720-47c2-8872-b43b59e8886d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:08.034657 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa07183-7720-47c2-8872-b43b59e8886d None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:08.047079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 407/825] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:35:07 2022] GET /identity/v3/role_assignments?user.id=5d4a107ed04348b59d20d1703dbe04eb&scope.project.id=59333e4f593346e184877d6bd9d22026&role.id=49bd5abf255048a0abaa9538e309f516 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:08.047079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa07183-7720-47c2-8872-b43b59e8886d None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:08.100363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6f24e-71e4-4ae9-a7cd-96fda596dccb None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:08.160556 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:08.172011 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd8887-8e5a-4f8c-9869-21a7bab64b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:08.173018 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd8887-8e5a-4f8c-9869-21a7bab64b6d None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:08.173301 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd8887-8e5a-4f8c-9869-21a7bab64b6d None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:08.173578 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd8887-8e5a-4f8c-9869-21a7bab64b6d None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:08.174305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd8887-8e5a-4f8c-9869-21a7bab64b6d None admin] RBAC: Adding query filter params (type=network) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:08.174618 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd8887-8e5a-4f8c-9869-21a7bab64b6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:08.176061 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd8887-8e5a-4f8c-9869-21a7bab64b6d None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:08.187257 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 419/826] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Jul 20 14:35:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:08.193153 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd8887-8e5a-4f8c-9869-21a7bab64b6d None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:08.203482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:08.246582 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa07183-7720-47c2-8872-b43b59e8886d None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:08.289321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aaaad5-428b-4459-9333-d08e25fadd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:08.291093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aaaad5-428b-4459-9333-d08e25fadd96 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:08.291526 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aaaad5-428b-4459-9333-d08e25fadd96 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:08.292072 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aaaad5-428b-4459-9333-d08e25fadd96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:08.308737 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aaaad5-428b-4459-9333-d08e25fadd96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:08.311420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aaaad5-428b-4459-9333-d08e25fadd96 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:08.332938 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd8887-8e5a-4f8c-9869-21a7bab64b6d None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:08.336304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-e7aaaad5-428b-4459-9333-d08e25fadd96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:08.336304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:08.336304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:08.336304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:08.336304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:08.336304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:08.336304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:08.336304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:08.336304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:08.336304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:08.336304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:08.336304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:08.336304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:35:08.336304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:35:08.336304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:35:08.337191 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:35:08.337191 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:08.337191 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:08.337191 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:08.337191 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:08.337191 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:08.337191 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:08.337191 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:08.337191 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:08.337191 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:08.337191 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:08.337191 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:08.337191 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:08.337191 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:08.337191 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:08.337191 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:08.337191 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:08.338343 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:35:08.338343 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:35:08.338343 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:08.338343 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:08.340336 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 408/827] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:08.348807 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aaaad5-428b-4459-9333-d08e25fadd96 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:08.376865 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52022d-44b0-41dc-a01e-e8324e00aaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:08.377454 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52022d-44b0-41dc-a01e-e8324e00aaed None admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:08.377742 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52022d-44b0-41dc-a01e-e8324e00aaed None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:08.378290 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52022d-44b0-41dc-a01e-e8324e00aaed None admin] PATH_INFO: `/v3/endpoints` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:08.378761 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e52022d-44b0-41dc-a01e-e8324e00aaed None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:08.381138 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e52022d-44b0-41dc-a01e-e8324e00aaed None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:08.406914 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 420/828] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Jul 20 14:35:08 2022] POST /identity/v3/endpoints => generated 327 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:35:08.484207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aaaad5-428b-4459-9333-d08e25fadd96 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:08.516032 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:08.541427 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9940107-5bef-4ce8-a74e-bb958b28ddd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:08.542122 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9940107-5bef-4ce8-a74e-bb958b28ddd0 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:08.542413 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9940107-5bef-4ce8-a74e-bb958b28ddd0 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:08.542683 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9940107-5bef-4ce8-a74e-bb958b28ddd0 None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:08.543531 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9940107-5bef-4ce8-a74e-bb958b28ddd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:08.543842 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9940107-5bef-4ce8-a74e-bb958b28ddd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:08.545295 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9940107-5bef-4ce8-a74e-bb958b28ddd0 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:08.564534 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 409/829] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:35:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:08.570586 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52022d-44b0-41dc-a01e-e8324e00aaed None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:08.640637 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:08.698972 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52022d-44b0-41dc-a01e-e8324e00aaed None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:08.740461 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f20de1-012b-4981-9ea0-ca9faba34c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:08.741685 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f20de1-012b-4981-9ea0-ca9faba34c51 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:08.742177 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f20de1-012b-4981-9ea0-ca9faba34c51 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:08.742681 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f20de1-012b-4981-9ea0-ca9faba34c51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:08.752190 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f20de1-012b-4981-9ea0-ca9faba34c51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:08.754007 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f20de1-012b-4981-9ea0-ca9faba34c51 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:08.770155 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-33f20de1-012b-4981-9ea0-ca9faba34c51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:35:08.770155 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:08.770155 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:08.770155 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:08.770155 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:08.770155 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:08.770155 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:08.770155 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:08.770155 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:08.770155 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:08.770155 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:08.770155 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:08.770155 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:35:08.770155 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:35:08.770155 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:35:08.771207 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:35:08.771207 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:08.771207 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:08.771207 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:08.771207 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:08.771207 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:08.771207 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:08.771207 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:08.771207 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:08.771207 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:08.771207 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:08.771207 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:08.771207 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:08.771207 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:08.771207 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:08.771207 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:08.771207 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:08.772138 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:35:08.772138 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:35:08.772138 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:35:08.772138 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:35:08.772138 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:35:08.772138 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:35:08.772138 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:35:08.772138 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:08.774594 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 421/830] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:35:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:08.784185 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9940107-5bef-4ce8-a74e-bb958b28ddd0 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:08.902604 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9940107-5bef-4ce8-a74e-bb958b28ddd0 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:08.960112 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6aa20-074a-4660-832b-0f329a5eacad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:08.961126 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6aa20-074a-4660-832b-0f329a5eacad None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:08.961579 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6aa20-074a-4660-832b-0f329a5eacad None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:08.962019 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6aa20-074a-4660-832b-0f329a5eacad None admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:08.963002 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c6aa20-074a-4660-832b-0f329a5eacad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:08.963379 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c6aa20-074a-4660-832b-0f329a5eacad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:08.965167 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c6aa20-074a-4660-832b-0f329a5eacad None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:08.981245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 410/831] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:35:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:08.986676 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f20de1-012b-4981-9ea0-ca9faba34c51 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:09.091556 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f20de1-012b-4981-9ea0-ca9faba34c51 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:09.103407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:09.129919 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb759c-d811-40aa-8e4d-264b2a03799c None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:09.130276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb759c-d811-40aa-8e4d-264b2a03799c None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:09.130485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb759c-d811-40aa-8e4d-264b2a03799c None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:09.131886 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484a8ac-c8e0-4a50-b62d-90e79d19efc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:09.132143 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 411/832] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:09.133244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484a8ac-c8e0-4a50-b62d-90e79d19efc2 None admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:09.133788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484a8ac-c8e0-4a50-b62d-90e79d19efc2 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:09.134281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484a8ac-c8e0-4a50-b62d-90e79d19efc2 None admin] PATH_INFO: `/v3/projects/59333e4f593346e184877d6bd9d22026/users/2f8cf6d908314281a3ddc9d9513d39c2/roles/62e9fa78a7c44a16bb4540d3e8d7044d` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:09.137310 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efbf5c-0e20-491f-b5b6-b5a9c83e5d82 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:09.137602 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efbf5c-0e20-491f-b5b6-b5a9c83e5d82 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:09.137879 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efbf5c-0e20-491f-b5b6-b5a9c83e5d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:09.152585 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d484a8ac-c8e0-4a50-b62d-90e79d19efc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=59333e4f593346e184877d6bd9d22026, user_id=2f8cf6d908314281a3ddc9d9513d39c2, role_id=62e9fa78a7c44a16bb4540d3e8d7044d)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:09.154410 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-90efbf5c-0e20-491f-b5b6-b5a9c83e5d82 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:09.154666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d484a8ac-c8e0-4a50-b62d-90e79d19efc2 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:09.155610 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-90efbf5c-0e20-491f-b5b6-b5a9c83e5d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:09.174712 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 422/833] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jul 20 14:35:08 2022] PUT /identity/v3/projects/59333e4f593346e184877d6bd9d22026/users/2f8cf6d908314281a3ddc9d9513d39c2/roles/62e9fa78a7c44a16bb4540d3e8d7044d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:35:09.203237 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:09.241915 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 412/834] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:09 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 14:35:09.266502 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484a8ac-c8e0-4a50-b62d-90e79d19efc2 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:09.290507 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70643b71-ecd3-46c5-b13e-1ce437468564 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:09.290828 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70643b71-ecd3-46c5-b13e-1ce437468564 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:09.291171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70643b71-ecd3-46c5-b13e-1ce437468564 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:09.292793 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 413/835] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:09.300214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247063e-ee80-4a86-8e01-204894830c8f None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:09.300727 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247063e-ee80-4a86-8e01-204894830c8f None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:09.301054 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247063e-ee80-4a86-8e01-204894830c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:09.320306 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-9247063e-ee80-4a86-8e01-204894830c8f None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:09.321638 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-9247063e-ee80-4a86-8e01-204894830c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:09.385378 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484a8ac-c8e0-4a50-b62d-90e79d19efc2 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:09.399121 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 414/836] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:09 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 14:35:09.426363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e77aa-1afe-4db9-b435-34c71978920b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:09.427321 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e77aa-1afe-4db9-b435-34c71978920b None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:09.427762 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e77aa-1afe-4db9-b435-34c71978920b None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:09.428090 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e77aa-1afe-4db9-b435-34c71978920b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:09.428620 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247063e-ee80-4a86-8e01-204894830c8f None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:09.439955 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e77aa-1afe-4db9-b435-34c71978920b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:35:09.448077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-271e77aa-1afe-4db9-b435-34c71978920b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:35:09.448077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:09.448077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:09.448077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:09.448077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:09.448077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:09.448077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:09.448077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:09.448077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:09.448077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:09.448077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:09.448077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:09.448077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:35:09.448077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:35:09.448077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:35:09.448077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:35:09.449022 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:35:09.449022 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:35:09.449022 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:35:09.449022 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:35:09.449022 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:09.449022 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:09.449022 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:09.449022 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:09.449022 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:09.449022 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:09.449022 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:09.449022 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:09.449022 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:09.449022 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:09.449022 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:09.449022 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:09.449876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:09.449876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:09.449876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:09.449876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:09.449876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:35:09.449876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:35:09.449876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:35:09.449876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:35:09.449876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:35:09.449876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:35:09.449876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:35:09.449876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:09.449876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 423/837] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:35:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:09.455460 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e77aa-1afe-4db9-b435-34c71978920b None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:09.479326 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:09.561837 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247063e-ee80-4a86-8e01-204894830c8f None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:09.564805 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:09.565696 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e77aa-1afe-4db9-b435-34c71978920b None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:09.606531 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e9995-9393-40c8-b04b-3be4352f9142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:09.607359 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e9995-9393-40c8-b04b-3be4352f9142 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:09.607721 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e9995-9393-40c8-b04b-3be4352f9142 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:09.608070 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e9995-9393-40c8-b04b-3be4352f9142 None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:09.608923 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01e9995-9393-40c8-b04b-3be4352f9142 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:09.609352 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01e9995-9393-40c8-b04b-3be4352f9142 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:09.610906 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01e9995-9393-40c8-b04b-3be4352f9142 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:09.615582 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af29ea-02cd-4d9d-bfb2-8a966478644d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:09.617152 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af29ea-02cd-4d9d-bfb2-8a966478644d None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:09.617615 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af29ea-02cd-4d9d-bfb2-8a966478644d None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:09.618336 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af29ea-02cd-4d9d-bfb2-8a966478644d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:09.619382 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af29ea-02cd-4d9d-bfb2-8a966478644d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:09.621861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af29ea-02cd-4d9d-bfb2-8a966478644d None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:09.624989 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 424/838] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:35:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:09.625881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e9995-9393-40c8-b04b-3be4352f9142 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:09.637171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-a1af29ea-02cd-4d9d-bfb2-8a966478644d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 14:35:09.637171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:09.637171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 14:35:09.637171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 14:35:09.637171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 14:35:09.637171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 14:35:09.637171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 14:35:09.637171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:09.637171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 14:35:09.637171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:09.637171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 14:35:09.637171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:09.637171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:09.637171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:09.637171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:09.637171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:09.637171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:09.638196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:09.638196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:09.638196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:09.638196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:09.638196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:09.638196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:09.638196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 14:35:09.638196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 14:35:09.638196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 14:35:09.638196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 14:35:09.638196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:09.638196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:09.638196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:09.638196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:09.638196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:09.638196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:09.639029 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:09.639029 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:09.639029 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:09.639029 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:09.639029 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:09.639029 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:09.639029 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:09.639029 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:09.639029 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:09.639029 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:09.639029 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 14:35:09.639029 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:09.639029 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 14:35:09.639029 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:09.645654 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 415/839] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jul 20 14:35:09 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:35:09.646376 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af29ea-02cd-4d9d-bfb2-8a966478644d None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:09.739277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:09.760279 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af29ea-02cd-4d9d-bfb2-8a966478644d None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:09.765981 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e9995-9393-40c8-b04b-3be4352f9142 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:09.795123 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f09be5-9625-4c75-bb3e-26da89753351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:09.796037 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f09be5-9625-4c75-bb3e-26da89753351 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:09.796494 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f09be5-9625-4c75-bb3e-26da89753351 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:09.796913 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f09be5-9625-4c75-bb3e-26da89753351 None admin] PATH_INFO: `/v3/services` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:09.798116 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f09be5-9625-4c75-bb3e-26da89753351 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:09.798567 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f09be5-9625-4c75-bb3e-26da89753351 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:09.800176 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f09be5-9625-4c75-bb3e-26da89753351 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:09.805455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 416/840] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jul 20 14:35:09 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:09.810922 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f09be5-9625-4c75-bb3e-26da89753351 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:09.818156 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea2163d-fb65-4d2e-a176-12b12bf4df6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:09.819327 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2163d-fb65-4d2e-a176-12b12bf4df6b None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:09.820316 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2163d-fb65-4d2e-a176-12b12bf4df6b None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:09.820813 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2163d-fb65-4d2e-a176-12b12bf4df6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:09.835847 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea2163d-fb65-4d2e-a176-12b12bf4df6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:09.838249 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea2163d-fb65-4d2e-a176-12b12bf4df6b None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:09.842979 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:09.863253 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-4ea2163d-fb65-4d2e-a176-12b12bf4df6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:09.863253 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:09.863253 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:09.863253 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:09.863253 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:09.863253 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:09.863253 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:09.863253 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:09.863253 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:09.863253 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:09.863253 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:09.863253 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:09.863253 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:35:09.863253 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:35:09.863253 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:35:09.864245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:35:09.864245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:09.864245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:09.864245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:09.864245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:09.864245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:09.864245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:09.864245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:09.864245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:09.864245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:09.864245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:09.864245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:09.864245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:09.864245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:09.864245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:09.864245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:09.864245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:09.865213 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:35:09.865213 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:35:09.865213 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:09.865213 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:09.868123 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 425/841] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:09.876768 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea2163d-fb65-4d2e-a176-12b12bf4df6b None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:09.919889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f09be5-9625-4c75-bb3e-26da89753351 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:09.957735 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bbffff-4124-45be-ba21-50038ffe6a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:09.958554 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbffff-4124-45be-ba21-50038ffe6a94 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:09.959073 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbffff-4124-45be-ba21-50038ffe6a94 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:09.959565 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbffff-4124-45be-ba21-50038ffe6a94 None admin] PATH_INFO: `/v3/services` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:09.960517 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bbffff-4124-45be-ba21-50038ffe6a94 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:09.960922 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bbffff-4124-45be-ba21-50038ffe6a94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:09.962397 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bbffff-4124-45be-ba21-50038ffe6a94 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:09.967786 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 417/842] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jul 20 14:35:09 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:09.999400 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea2163d-fb65-4d2e-a176-12b12bf4df6b None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:10.036780 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa24e40-cc28-4542-8dae-1d3986cfd97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:10.037527 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa24e40-cc28-4542-8dae-1d3986cfd97c None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:10.037815 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa24e40-cc28-4542-8dae-1d3986cfd97c None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:10.038098 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa24e40-cc28-4542-8dae-1d3986cfd97c None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:10.038847 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa24e40-cc28-4542-8dae-1d3986cfd97c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:10.039197 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa24e40-cc28-4542-8dae-1d3986cfd97c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:10.040668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa24e40-cc28-4542-8dae-1d3986cfd97c None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:10.042444 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:10.056745 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 426/843] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:35:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:10.062484 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bbffff-4124-45be-ba21-50038ffe6a94 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:10.169993 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:10.178197 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bbffff-4124-45be-ba21-50038ffe6a94 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:10.213632 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eece4f-6fbe-4733-9343-17da0a90188c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:10.214722 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eece4f-6fbe-4733-9343-17da0a90188c None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:10.215172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eece4f-6fbe-4733-9343-17da0a90188c None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:10.215581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eece4f-6fbe-4733-9343-17da0a90188c None admin] PATH_INFO: `/v3/users/swift` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:10.223552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eece4f-6fbe-4733-9343-17da0a90188c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:10.225356 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eece4f-6fbe-4733-9343-17da0a90188c None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:10.239525 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-33eece4f-6fbe-4733-9343-17da0a90188c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 14:35:10.239525 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:10.239525 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:10.239525 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:10.239525 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:10.239525 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:10.239525 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:10.239525 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:10.239525 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:10.239525 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:10.239525 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:10.239525 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:10.239525 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:35:10.239525 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:35:10.239525 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:35:10.239525 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:35:10.240758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:10.240758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:10.240758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:35:10.240758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:35:10.240758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:35:10.240758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:35:10.240758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:10.240758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:10.240758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:10.240758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:10.240758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:10.240758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:10.240758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:10.240758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:10.240758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:10.240758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:10.240758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:10.240758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:10.241951 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:10.241951 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:10.241951 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:35:10.241951 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:35:10.241951 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:35:10.241951 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:35:10.241951 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:35:10.241951 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:35:10.241951 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 14:35:10.241951 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:10.244681 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 418/844] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:35:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:10.250547 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa24e40-cc28-4542-8dae-1d3986cfd97c None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:10.353355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa24e40-cc28-4542-8dae-1d3986cfd97c None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:10.401087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f039766b-57bb-40e4-a8b1-ceec294331f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:10.401800 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039766b-57bb-40e4-a8b1-ceec294331f9 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:10.402055 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039766b-57bb-40e4-a8b1-ceec294331f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:10.402374 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039766b-57bb-40e4-a8b1-ceec294331f9 None admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:10.403770 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f039766b-57bb-40e4-a8b1-ceec294331f9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:10.404240 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f039766b-57bb-40e4-a8b1-ceec294331f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:10.406507 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f039766b-57bb-40e4-a8b1-ceec294331f9 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:10.431298 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 427/845] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Jul 20 14:35:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:10.436911 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eece4f-6fbe-4733-9343-17da0a90188c None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:10.506812 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34c9b5-f54e-4b82-9afe-7f013456dda2 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:10.507167 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34c9b5-f54e-4b82-9afe-7f013456dda2 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:10.507387 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34c9b5-f54e-4b82-9afe-7f013456dda2 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:10.509294 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 428/846] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 14:35:10.514502 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7365e-a6a2-47f5-b9ed-4275552bc9eb None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:10.514794 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7365e-a6a2-47f5-b9ed-4275552bc9eb None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:10.515113 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7365e-a6a2-47f5-b9ed-4275552bc9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:10.532968 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-e2f7365e-a6a2-47f5-b9ed-4275552bc9eb None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:10.534303 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e2f7365e-a6a2-47f5-b9ed-4275552bc9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:10.554976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eece4f-6fbe-4733-9343-17da0a90188c None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:10.589138 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:10.599441 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7622d-fce8-4b28-a908-35623ec574fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:10.600651 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7622d-fce8-4b28-a908-35623ec574fd None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:10.601548 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7622d-fce8-4b28-a908-35623ec574fd None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:10.602287 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7622d-fce8-4b28-a908-35623ec574fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:10.619414 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 429/847] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:10 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 14:35:10.629186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:10.635185 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7365e-a6a2-47f5-b9ed-4275552bc9eb None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:10.687213 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c7622d-fce8-4b28-a908-35623ec574fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:10.688998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c7622d-fce8-4b28-a908-35623ec574fd None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:10.752122 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7365e-a6a2-47f5-b9ed-4275552bc9eb None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:10.788867 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de24aa43-c6d3-41f2-b807-c0caa25888b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:10.790074 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24aa43-c6d3-41f2-b807-c0caa25888b6 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:10.790564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24aa43-c6d3-41f2-b807-c0caa25888b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:10.791041 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24aa43-c6d3-41f2-b807-c0caa25888b6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:10.791934 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de24aa43-c6d3-41f2-b807-c0caa25888b6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:10.793571 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de24aa43-c6d3-41f2-b807-c0caa25888b6 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:10.810436 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-de24aa43-c6d3-41f2-b807-c0caa25888b6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 14:35:10.810436 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:10.810436 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 14:35:10.810436 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 14:35:10.810436 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 14:35:10.810436 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 14:35:10.810436 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 14:35:10.810436 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:10.810436 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 14:35:10.810436 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:10.810436 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 14:35:10.810436 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:10.810436 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:10.810436 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:10.810436 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:10.810436 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:10.811497 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:10.811497 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:10.811497 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:10.811497 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:10.811497 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:10.811497 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:10.811497 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:10.811497 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 14:35:10.811497 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 14:35:10.811497 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 14:35:10.811497 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 14:35:10.811497 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:10.811497 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:10.811497 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:10.811497 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:10.811497 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:10.812343 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:10.812343 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:10.812343 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:10.812343 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:10.812343 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:10.812343 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:10.812343 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:10.812343 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:10.812343 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:10.812343 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:10.812343 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:10.812343 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 14:35:10.812343 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:10.812343 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 14:35:10.812343 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:10.815193 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 430/848] 10.222.0.15 () {56 vars in 1132 bytes} [Wed Jul 20 14:35:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:10.818527 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de24aa43-c6d3-41f2-b807-c0caa25888b6 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:10.823088 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 419/849] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:35:10 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 14:35:10.829170 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7622d-fce8-4b28-a908-35623ec574fd None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:10.880454 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:10.899563 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:10.940232 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de24aa43-c6d3-41f2-b807-c0caa25888b6 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:10.948983 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7622d-fce8-4b28-a908-35623ec574fd None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:10.976455 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07695a14-8bf4-4a70-be43-2b922ca46e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:10.978062 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07695a14-8bf4-4a70-be43-2b922ca46e3d None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:10.978830 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07695a14-8bf4-4a70-be43-2b922ca46e3d None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:10.979369 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07695a14-8bf4-4a70-be43-2b922ca46e3d None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:10.980460 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07695a14-8bf4-4a70-be43-2b922ca46e3d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:10.981245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07695a14-8bf4-4a70-be43-2b922ca46e3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:10.983296 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07695a14-8bf4-4a70-be43-2b922ca46e3d None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:10.985691 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fdc36e-be19-436b-9ad4-6112446c3f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:10.986579 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fdc36e-be19-436b-9ad4-6112446c3f22 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:10.986950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fdc36e-be19-436b-9ad4-6112446c3f22 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:10.987330 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fdc36e-be19-436b-9ad4-6112446c3f22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:10.993889 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 431/850] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Jul 20 14:35:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:10.995745 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07695a14-8bf4-4a70-be43-2b922ca46e3d None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:11.004461 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fdc36e-be19-436b-9ad4-6112446c3f22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:11.005979 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fdc36e-be19-436b-9ad4-6112446c3f22 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:11.025208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-a5fdc36e-be19-436b-9ad4-6112446c3f22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:11.025208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:11.025208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:11.025208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:11.025208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:11.025208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:11.025208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:11.025208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:11.025208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:11.025208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:11.025208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:11.025208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:11.025208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:35:11.025208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:35:11.025208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:35:11.026207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:35:11.026207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:11.026207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:11.026207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:11.026207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:11.026207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:11.026207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:11.026207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:11.026207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:11.026207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:11.026207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:11.026207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:11.026207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:11.026207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:11.026207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:11.026207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:11.026207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:11.027571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:35:11.027571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:35:11.027571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:11.027571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:11.031538 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 420/851] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:11.032689 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fdc36e-be19-436b-9ad4-6112446c3f22 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:11.111751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07695a14-8bf4-4a70-be43-2b922ca46e3d None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:11.124589 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:11.141124 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fdc36e-be19-436b-9ad4-6112446c3f22 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:11.150209 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:11.152854 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe174d17-1fbf-4565-a423-48446d8b2109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:11.153835 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe174d17-1fbf-4565-a423-48446d8b2109 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:11.154296 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe174d17-1fbf-4565-a423-48446d8b2109 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:11.155240 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe174d17-1fbf-4565-a423-48446d8b2109 None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:11.156210 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe174d17-1fbf-4565-a423-48446d8b2109 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:11.156764 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe174d17-1fbf-4565-a423-48446d8b2109 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:11.158416 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe174d17-1fbf-4565-a423-48446d8b2109 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:11.167625 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 432/852] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Jul 20 14:35:10 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:11.182519 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b5b5a-04cb-4596-9a27-ebc13fce34c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:11.183252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b5b5a-04cb-4596-9a27-ebc13fce34c6 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:11.183539 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b5b5a-04cb-4596-9a27-ebc13fce34c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:11.183819 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b5b5a-04cb-4596-9a27-ebc13fce34c6 None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:11.184633 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b5b5a-04cb-4596-9a27-ebc13fce34c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:11.184947 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b5b5a-04cb-4596-9a27-ebc13fce34c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:11.186393 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b5b5a-04cb-4596-9a27-ebc13fce34c6 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:11.200827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 421/853] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:35:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:11.206696 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe174d17-1fbf-4565-a423-48446d8b2109 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:11.357312 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ef450-ec51-4fae-99ed-040d253c6fd2 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:11.358322 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ef450-ec51-4fae-99ed-040d253c6fd2 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:11.358831 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ef450-ec51-4fae-99ed-040d253c6fd2 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:11.360584 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 422/854] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:11.366262 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ba851-4bab-4fea-a8d5-ff57823594ee None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:11.366489 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ba851-4bab-4fea-a8d5-ff57823594ee None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:11.366768 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ba851-4bab-4fea-a8d5-ff57823594ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:11.386594 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-069ba851-4bab-4fea-a8d5-ff57823594ee None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:11.387921 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-069ba851-4bab-4fea-a8d5-ff57823594ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:11.440851 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe174d17-1fbf-4565-a423-48446d8b2109 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:11.459095 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 423/855] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:11 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 14:35:11.477134 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74978a75-d181-4eb2-99d9-6c68d5547dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:11.478038 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74978a75-d181-4eb2-99d9-6c68d5547dbb None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:11.478325 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74978a75-d181-4eb2-99d9-6c68d5547dbb None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:11.478603 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74978a75-d181-4eb2-99d9-6c68d5547dbb None admin] PATH_INFO: `/v3/projects/service` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:11.482586 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ba851-4bab-4fea-a8d5-ff57823594ee None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:11.488435 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74978a75-d181-4eb2-99d9-6c68d5547dbb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:11.489930 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74978a75-d181-4eb2-99d9-6c68d5547dbb None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:11.505313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-74978a75-d181-4eb2-99d9-6c68d5547dbb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:35:11.505313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:11.505313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:11.505313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:11.505313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:11.505313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:11.505313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:11.505313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:11.505313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:11.505313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:11.505313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:11.505313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:11.505313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:35:11.505313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:35:11.505313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:35:11.506285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:35:11.506285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:11.506285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:11.506285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:11.506285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:11.506285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:11.506285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:11.506285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:11.506285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:11.506285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:11.506285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:11.506285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:11.506285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:11.506285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:11.506285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:11.506285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:11.506285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:11.507208 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:35:11.507208 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:35:11.507208 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:35:11.507208 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:35:11.507208 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:35:11.507208 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:35:11.507208 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:35:11.507208 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:11.507208 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 433/856] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:35:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:11.513532 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74978a75-d181-4eb2-99d9-6c68d5547dbb None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:11.556622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:11.602293 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:11.610423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ba851-4bab-4fea-a8d5-ff57823594ee None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:11.643089 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74978a75-d181-4eb2-99d9-6c68d5547dbb None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:11.657900 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6acc15-4856-4e1b-80bb-47a96f93f920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:11.659095 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6acc15-4856-4e1b-80bb-47a96f93f920 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:11.659550 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6acc15-4856-4e1b-80bb-47a96f93f920 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:11.660016 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6acc15-4856-4e1b-80bb-47a96f93f920 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:11.671367 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6acc15-4856-4e1b-80bb-47a96f93f920 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:35:11.680266 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-cc6acc15-4856-4e1b-80bb-47a96f93f920 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:35:11.680266 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:11.680266 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:11.680266 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:11.680266 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:11.680266 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:11.680266 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:11.680266 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:11.680266 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:11.680266 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:11.680266 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:11.680266 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:11.680266 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:35:11.680266 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:35:11.680266 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:35:11.680266 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:35:11.681186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:35:11.681186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:35:11.681186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:35:11.681186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:35:11.681186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:11.681186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:11.681186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:11.681186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:11.681186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:11.681186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:11.681186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:11.681186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:11.681186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:11.681186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:11.681186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:11.681186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:11.682022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:11.682022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:11.682022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:11.682022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:11.682022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:35:11.682022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:35:11.682022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:35:11.682022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:35:11.682022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:35:11.682022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:35:11.682022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:35:11.682022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:11.689542 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca934a6-37cc-4d37-b69a-845dd9f992a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:11.689542 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca934a6-37cc-4d37-b69a-845dd9f992a6 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:11.689542 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca934a6-37cc-4d37-b69a-845dd9f992a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:11.689542 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca934a6-37cc-4d37-b69a-845dd9f992a6 None admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:11.689924 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca934a6-37cc-4d37-b69a-845dd9f992a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:11.689924 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca934a6-37cc-4d37-b69a-845dd9f992a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:11.689924 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca934a6-37cc-4d37-b69a-845dd9f992a6 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:11.690518 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 424/857] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jul 20 14:35:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:11.691645 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6acc15-4856-4e1b-80bb-47a96f93f920 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:11.707184 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 434/858] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:35:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:11.709470 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca934a6-37cc-4d37-b69a-845dd9f992a6 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:11.798879 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6acc15-4856-4e1b-80bb-47a96f93f920 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:11.817593 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:11.825472 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca934a6-37cc-4d37-b69a-845dd9f992a6 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:11.835575 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44277fb9-cf41-432c-accd-a80539e48fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:11.836597 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44277fb9-cf41-432c-accd-a80539e48fac None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:11.837077 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44277fb9-cf41-432c-accd-a80539e48fac None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:11.837547 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44277fb9-cf41-432c-accd-a80539e48fac None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:11.838468 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44277fb9-cf41-432c-accd-a80539e48fac None admin] RBAC: Adding query filter params (name=service) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:11.838943 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44277fb9-cf41-432c-accd-a80539e48fac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:11.840668 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44277fb9-cf41-432c-accd-a80539e48fac None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:11.854794 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 425/859] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jul 20 14:35:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:11.856173 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44277fb9-cf41-432c-accd-a80539e48fac None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:11.861401 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adc9b6-a0ee-4952-b600-495475b63e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:11.862061 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adc9b6-a0ee-4952-b600-495475b63e92 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:11.862413 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adc9b6-a0ee-4952-b600-495475b63e92 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:11.862759 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adc9b6-a0ee-4952-b600-495475b63e92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:11.863710 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67adc9b6-a0ee-4952-b600-495475b63e92 None admin] RBAC: Adding query filter params (role.id=49bd5abf255048a0abaa9538e309f516, scope.project.id=59333e4f593346e184877d6bd9d22026, user.id=37f4844988474d7a91133d77906611a5) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:11.864157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67adc9b6-a0ee-4952-b600-495475b63e92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:11.865771 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67adc9b6-a0ee-4952-b600-495475b63e92 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:11.879227 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 435/860] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:35:11 2022] GET /identity/v3/role_assignments?user.id=37f4844988474d7a91133d77906611a5&scope.project.id=59333e4f593346e184877d6bd9d22026&role.id=49bd5abf255048a0abaa9538e309f516 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:11.879668 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:11.974854 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44277fb9-cf41-432c-accd-a80539e48fac None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:12.013204 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb56a1-3391-47bf-a273-a0e3398c8185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:12.014344 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb56a1-3391-47bf-a273-a0e3398c8185 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:12.014809 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb56a1-3391-47bf-a273-a0e3398c8185 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:12.015297 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb56a1-3391-47bf-a273-a0e3398c8185 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:12.028694 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb56a1-3391-47bf-a273-a0e3398c8185 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:12.030462 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb56a1-3391-47bf-a273-a0e3398c8185 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:12.049219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-01eb56a1-3391-47bf-a273-a0e3398c8185 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:12.049219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:12.049219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:12.049219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:12.049219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:12.049219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:12.049219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:12.049219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:12.049219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:12.049219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:12.049219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:12.049219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:12.049219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:35:12.049219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:35:12.049219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:35:12.050168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:35:12.050168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:12.050168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:12.050168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:12.050168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:12.050168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:12.050168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:12.050168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:12.050168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:12.050168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:12.050168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:12.050168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:12.050168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:12.050168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:12.050168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:12.050168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:12.050168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:12.051075 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:35:12.051075 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:35:12.051075 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:12.051075 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:12.051637 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 426/861] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:12.061796 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adc9b6-a0ee-4952-b600-495475b63e92 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:12.107705 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dead00-7016-4615-b4bd-2e10a14dda15 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:12.108036 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dead00-7016-4615-b4bd-2e10a14dda15 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:12.108314 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dead00-7016-4615-b4bd-2e10a14dda15 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:12.114950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 427/862] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:12.116872 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ede9c-09d6-4ec8-b028-eb11fe183acc None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:12.117161 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ede9c-09d6-4ec8-b028-eb11fe183acc None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:12.117436 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ede9c-09d6-4ec8-b028-eb11fe183acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:12.136515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-d09ede9c-09d6-4ec8-b028-eb11fe183acc None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:12.137821 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-d09ede9c-09d6-4ec8-b028-eb11fe183acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:12.182852 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adc9b6-a0ee-4952-b600-495475b63e92 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:12.207843 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:12.219273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 428/863] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 14:35:12.226419 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d90cb5-5591-4a90-b2bb-5f3e747b6f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:12.227363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d90cb5-5591-4a90-b2bb-5f3e747b6f06 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:12.227865 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d90cb5-5591-4a90-b2bb-5f3e747b6f06 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:12.228348 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d90cb5-5591-4a90-b2bb-5f3e747b6f06 None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:12.229264 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d90cb5-5591-4a90-b2bb-5f3e747b6f06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:12.229771 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d90cb5-5591-4a90-b2bb-5f3e747b6f06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:12.231446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d90cb5-5591-4a90-b2bb-5f3e747b6f06 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:12.237832 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ede9c-09d6-4ec8-b028-eb11fe183acc None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:12.247446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 436/864] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:35:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:12.259092 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d90cb5-5591-4a90-b2bb-5f3e747b6f06 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:12.274737 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:12.362767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d90cb5-5591-4a90-b2bb-5f3e747b6f06 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:12.368065 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ede9c-09d6-4ec8-b028-eb11fe183acc None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:12.401122 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a622c1e-699a-4c59-8a66-8a8ae0ab0306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:12.401969 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a622c1e-699a-4c59-8a66-8a8ae0ab0306 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:12.402253 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a622c1e-699a-4c59-8a66-8a8ae0ab0306 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:12.402532 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a622c1e-699a-4c59-8a66-8a8ae0ab0306 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:12.406095 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f65d73c-b62f-41bc-97a4-1381bceb619c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:12.407404 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65d73c-b62f-41bc-97a4-1381bceb619c None admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:12.407694 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65d73c-b62f-41bc-97a4-1381bceb619c None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:12.407975 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65d73c-b62f-41bc-97a4-1381bceb619c None admin] PATH_INFO: `/v3/services` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:12.408743 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f65d73c-b62f-41bc-97a4-1381bceb619c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:12.410231 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f65d73c-b62f-41bc-97a4-1381bceb619c None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:12.415635 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a622c1e-699a-4c59-8a66-8a8ae0ab0306 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:12.417474 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a622c1e-699a-4c59-8a66-8a8ae0ab0306 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:12.428713 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 429/865] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Jul 20 14:35:12 2022] POST /identity/v3/services => generated 251 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 14:35:12.439172 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-9a622c1e-699a-4c59-8a66-8a8ae0ab0306 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 14:35:12.439172 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:12.439172 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:12.439172 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:12.439172 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:12.439172 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:12.439172 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:12.439172 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:12.439172 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:12.439172 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:12.439172 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:12.439172 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:12.439172 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:35:12.439172 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:35:12.439172 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:35:12.439172 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:35:12.440206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:12.440206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:12.440206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:35:12.440206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:35:12.440206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:35:12.440206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:35:12.440206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:12.440206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:12.440206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:12.440206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:12.440206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:12.440206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:12.440206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:12.440206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:12.440206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:12.440206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:12.440206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:12.440206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:12.441917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:12.441917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:12.441917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:35:12.441917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:35:12.441917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:35:12.441917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:35:12.441917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:35:12.441917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:35:12.441917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 14:35:12.441917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:12.441917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 437/866] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Jul 20 14:35:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:12.444032 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f65d73c-b62f-41bc-97a4-1381bceb619c None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:12.542841 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:12.589236 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f65d73c-b62f-41bc-97a4-1381bceb619c None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:12.625280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9efb4-b1b0-46a8-9228-699b9e8af1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:12.625972 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9efb4-b1b0-46a8-9228-699b9e8af1de None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:12.626381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9efb4-b1b0-46a8-9228-699b9e8af1de None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:12.626663 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9efb4-b1b0-46a8-9228-699b9e8af1de None admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:12.627599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9efb4-b1b0-46a8-9228-699b9e8af1de None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:12.627913 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9efb4-b1b0-46a8-9228-699b9e8af1de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:12.629356 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9efb4-b1b0-46a8-9228-699b9e8af1de None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:12.648431 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 430/867] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Jul 20 14:35:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:12.654243 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a622c1e-699a-4c59-8a66-8a8ae0ab0306 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:12.738631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:12.761121 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a622c1e-699a-4c59-8a66-8a8ae0ab0306 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:12.795860 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119db7c3-91ca-4208-9b41-48850a04b939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:12.796520 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119db7c3-91ca-4208-9b41-48850a04b939 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:12.796806 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119db7c3-91ca-4208-9b41-48850a04b939 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:12.797138 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119db7c3-91ca-4208-9b41-48850a04b939 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:12.867298 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119db7c3-91ca-4208-9b41-48850a04b939 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:12.868821 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119db7c3-91ca-4208-9b41-48850a04b939 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:12.963885 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:12.967662 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 438/868] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:35:12 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 14:35:12.974677 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9efb4-b1b0-46a8-9228-699b9e8af1de None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:13.076671 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9efb4-b1b0-46a8-9228-699b9e8af1de None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:13.092983 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:13.114491 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298cc51e-44e9-4c66-a498-b71daec4f098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:13.115389 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298cc51e-44e9-4c66-a498-b71daec4f098 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:13.115683 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298cc51e-44e9-4c66-a498-b71daec4f098 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:13.115959 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298cc51e-44e9-4c66-a498-b71daec4f098 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:13.126751 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298cc51e-44e9-4c66-a498-b71daec4f098 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:13.128249 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298cc51e-44e9-4c66-a498-b71daec4f098 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:13.148693 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-298cc51e-44e9-4c66-a498-b71daec4f098 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:13.148693 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:13.148693 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:13.148693 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:13.148693 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:13.148693 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:13.148693 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:13.148693 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:13.148693 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:13.148693 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:13.148693 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:13.148693 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:13.148693 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:35:13.148693 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:35:13.148693 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:35:13.149634 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:35:13.149634 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:13.149634 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:13.149634 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:13.149634 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:13.149634 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:13.149634 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:13.149634 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:13.149634 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:13.149634 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:13.149634 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:13.149634 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:13.149634 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:13.149634 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:13.149634 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:13.149634 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:13.149634 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:13.150559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:35:13.150559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:35:13.150559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:13.150559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:13.150559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 431/869] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:13.156033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119db7c3-91ca-4208-9b41-48850a04b939 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:13.256342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119db7c3-91ca-4208-9b41-48850a04b939 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:13.290902 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413b361-8724-4dfd-8f16-610de6778a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:13.291986 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413b361-8724-4dfd-8f16-610de6778a3f None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:13.292497 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413b361-8724-4dfd-8f16-610de6778a3f None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:13.292984 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413b361-8724-4dfd-8f16-610de6778a3f None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:13.293912 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413b361-8724-4dfd-8f16-610de6778a3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:13.294412 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413b361-8724-4dfd-8f16-610de6778a3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:13.296018 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413b361-8724-4dfd-8f16-610de6778a3f None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:13.308505 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 439/870] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:35:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:13.319068 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298cc51e-44e9-4c66-a498-b71daec4f098 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:13.320636 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3ed99-5e53-4e8c-986c-a8d7b0153b12 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:13.321065 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3ed99-5e53-4e8c-986c-a8d7b0153b12 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:13.321358 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3ed99-5e53-4e8c-986c-a8d7b0153b12 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:13.322967 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 440/871] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:13.327996 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce839cff-ced2-4d66-b6b2-35ab67db106f None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:13.328234 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce839cff-ced2-4d66-b6b2-35ab67db106f None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:13.328519 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce839cff-ced2-4d66-b6b2-35ab67db106f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:13.345362 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-ce839cff-ced2-4d66-b6b2-35ab67db106f None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:13.346824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-ce839cff-ced2-4d66-b6b2-35ab67db106f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:13.382444 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:13.419272 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 441/872] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:13 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 14:35:13.421320 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298cc51e-44e9-4c66-a498-b71daec4f098 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:13.441932 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce839cff-ced2-4d66-b6b2-35ab67db106f None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:13.459340 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5066d5-4c0f-4223-ba21-ab3cf7683c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:13.460201 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5066d5-4c0f-4223-ba21-ab3cf7683c6a None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:13.460482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5066d5-4c0f-4223-ba21-ab3cf7683c6a None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:13.460759 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5066d5-4c0f-4223-ba21-ab3cf7683c6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:13.467956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5066d5-4c0f-4223-ba21-ab3cf7683c6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:13.469432 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5066d5-4c0f-4223-ba21-ab3cf7683c6a None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:13.496765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-1e5066d5-4c0f-4223-ba21-ab3cf7683c6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:35:13.496765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:13.496765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:13.496765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:13.496765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:13.496765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:13.496765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:13.496765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:13.496765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:13.496765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:13.496765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:13.496765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:13.496765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:35:13.496765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:35:13.496765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:35:13.497931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:35:13.497931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:13.497931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:13.497931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:13.497931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:13.497931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:13.497931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:13.497931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:13.497931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:13.497931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:13.497931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:13.497931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:13.497931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:13.497931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:13.497931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:13.497931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:13.497931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:13.499091 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:35:13.499091 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:35:13.499091 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:35:13.499091 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:35:13.499091 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:35:13.499091 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:35:13.499091 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:35:13.499091 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:13.500589 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 432/873] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:35:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:13.507518 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5066d5-4c0f-4223-ba21-ab3cf7683c6a None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:13.543204 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:13.558599 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce839cff-ced2-4d66-b6b2-35ab67db106f None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:13.598069 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9796f9-99a4-4bd7-af32-051ef089972a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:13.598667 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9796f9-99a4-4bd7-af32-051ef089972a None admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:13.598945 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9796f9-99a4-4bd7-af32-051ef089972a None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:13.599246 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9796f9-99a4-4bd7-af32-051ef089972a None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:13.600067 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9796f9-99a4-4bd7-af32-051ef089972a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:13.602286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9796f9-99a4-4bd7-af32-051ef089972a None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:13.619478 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 442/874] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Jul 20 14:35:13 2022] POST /identity/v3/services => generated 269 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 14:35:13.640719 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5066d5-4c0f-4223-ba21-ab3cf7683c6a None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:13.688912 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2cc76-ec24-418c-96e1-40ca5f961a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:13.690025 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2cc76-ec24-418c-96e1-40ca5f961a02 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:13.690515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2cc76-ec24-418c-96e1-40ca5f961a02 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:13.691011 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2cc76-ec24-418c-96e1-40ca5f961a02 None admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:13.692145 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2cc76-ec24-418c-96e1-40ca5f961a02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:13.692706 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2cc76-ec24-418c-96e1-40ca5f961a02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:13.694450 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2cc76-ec24-418c-96e1-40ca5f961a02 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:13.709834 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 433/875] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:35:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:13.720314 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9796f9-99a4-4bd7-af32-051ef089972a None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:13.760702 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:13.831113 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9796f9-99a4-4bd7-af32-051ef089972a None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:13.868676 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698026d0-1c2e-4b57-bca3-aecca59c5297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:13.869272 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698026d0-1c2e-4b57-bca3-aecca59c5297 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:13.869556 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698026d0-1c2e-4b57-bca3-aecca59c5297 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:13.869836 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698026d0-1c2e-4b57-bca3-aecca59c5297 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:13.870674 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698026d0-1c2e-4b57-bca3-aecca59c5297 None admin] RBAC: Adding query filter params (role.id=62e9fa78a7c44a16bb4540d3e8d7044d, scope.project.id=59333e4f593346e184877d6bd9d22026, user.id=2f8cf6d908314281a3ddc9d9513d39c2) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:13.870990 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698026d0-1c2e-4b57-bca3-aecca59c5297 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:13.872569 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698026d0-1c2e-4b57-bca3-aecca59c5297 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:13.878947 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 443/876] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:35:13 2022] GET /identity/v3/role_assignments?user.id=2f8cf6d908314281a3ddc9d9513d39c2&scope.project.id=59333e4f593346e184877d6bd9d22026&role.id=62e9fa78a7c44a16bb4540d3e8d7044d => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:14.082179 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e02142-c50b-4e3e-b7f9-fe3f7b714726 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:14.082861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e02142-c50b-4e3e-b7f9-fe3f7b714726 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:14.083375 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e02142-c50b-4e3e-b7f9-fe3f7b714726 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:14.085253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 434/877] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:14.090986 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707be08-09e0-4e98-9e37-83aeda47a05b None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:14.091565 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707be08-09e0-4e98-9e37-83aeda47a05b None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:14.092039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707be08-09e0-4e98-9e37-83aeda47a05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:14.110496 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-e707be08-09e0-4e98-9e37-83aeda47a05b None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:14.112044 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e707be08-09e0-4e98-9e37-83aeda47a05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:14.202269 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 444/878] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:14 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:35:14.231217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e02142-c50b-4e3e-b7f9-fe3f7b714726 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:14.319752 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:14.341426 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e02142-c50b-4e3e-b7f9-fe3f7b714726 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:14.376616 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad60d55-6284-4019-a9f7-99876103c272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:14.377703 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad60d55-6284-4019-a9f7-99876103c272 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:14.378163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad60d55-6284-4019-a9f7-99876103c272 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:14.378633 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad60d55-6284-4019-a9f7-99876103c272 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:14.379513 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad60d55-6284-4019-a9f7-99876103c272 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:14.381130 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad60d55-6284-4019-a9f7-99876103c272 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:14.393956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-7ad60d55-6284-4019-a9f7-99876103c272 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 14:35:14.393956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:14.393956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 14:35:14.393956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 14:35:14.393956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 14:35:14.393956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 14:35:14.393956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 14:35:14.393956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:14.393956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 14:35:14.393956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:14.393956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 14:35:14.393956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:14.393956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:14.393956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:14.393956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:14.393956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:14.393956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:14.395243 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:14.395243 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:14.395243 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:14.395243 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:14.395243 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:14.395243 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:14.395243 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 14:35:14.395243 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 14:35:14.395243 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 14:35:14.395243 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 14:35:14.395243 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:14.395243 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:14.395243 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:14.395243 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:14.395243 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:14.395243 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:14.396301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:14.396301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:14.396301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:14.396301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:14.396301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:14.396301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:14.396301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:14.396301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:14.396301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:14.396301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:14.396301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 14:35:14.396301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:14.396301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 14:35:14.396301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:14.398906 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 435/879] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Jul 20 14:35:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:14.408454 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707be08-09e0-4e98-9e37-83aeda47a05b None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:14.435262 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:14.519412 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707be08-09e0-4e98-9e37-83aeda47a05b None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:14.555987 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5fdcf1-e438-4196-ac5c-75919a6e8a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:14.556684 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fdcf1-e438-4196-ac5c-75919a6e8a8b None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:14.556978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fdcf1-e438-4196-ac5c-75919a6e8a8b None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:14.557261 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fdcf1-e438-4196-ac5c-75919a6e8a8b None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:14.557983 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5fdcf1-e438-4196-ac5c-75919a6e8a8b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:14.558296 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5fdcf1-e438-4196-ac5c-75919a6e8a8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:14.559713 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5fdcf1-e438-4196-ac5c-75919a6e8a8b None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:14.561707 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b874a-5b29-4510-a06d-ef292c07df98 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:14.562183 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b874a-5b29-4510-a06d-ef292c07df98 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:14.562271 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b874a-5b29-4510-a06d-ef292c07df98 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:14.564074 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 436/880] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:14.569263 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 445/881] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Jul 20 14:35:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:14.570881 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243216a-1a64-418b-9fe0-dac3b27ad288 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:14.571208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243216a-1a64-418b-9fe0-dac3b27ad288 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:14.571491 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243216a-1a64-418b-9fe0-dac3b27ad288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:14.574691 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5fdcf1-e438-4196-ac5c-75919a6e8a8b None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:14.592253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-d243216a-1a64-418b-9fe0-dac3b27ad288 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:14.593965 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-d243216a-1a64-418b-9fe0-dac3b27ad288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:14.670302 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:14.687080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 437/882] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:14 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 14:35:14.692454 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5fdcf1-e438-4196-ac5c-75919a6e8a8b None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:14.710135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d243216a-1a64-418b-9fe0-dac3b27ad288 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:14.732755 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949dc2ee-e63a-4ed9-8ad6-a3f76f798d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:14.733961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949dc2ee-e63a-4ed9-8ad6-a3f76f798d8f None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:14.734447 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949dc2ee-e63a-4ed9-8ad6-a3f76f798d8f None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:14.734922 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949dc2ee-e63a-4ed9-8ad6-a3f76f798d8f None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:14.735964 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949dc2ee-e63a-4ed9-8ad6-a3f76f798d8f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:14.736470 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949dc2ee-e63a-4ed9-8ad6-a3f76f798d8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:14.738065 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949dc2ee-e63a-4ed9-8ad6-a3f76f798d8f None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:14.744066 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 446/883] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Jul 20 14:35:14 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:14.805245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:14.832632 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d243216a-1a64-418b-9fe0-dac3b27ad288 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:14.872876 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e3cb6-4d07-42db-b2e2-ee267064e155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:14.874221 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e3cb6-4d07-42db-b2e2-ee267064e155 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:14.874771 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e3cb6-4d07-42db-b2e2-ee267064e155 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:14.875297 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e3cb6-4d07-42db-b2e2-ee267064e155 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:14.876177 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e3cb6-4d07-42db-b2e2-ee267064e155 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:14.877826 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e3cb6-4d07-42db-b2e2-ee267064e155 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:14.891245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-ba6e3cb6-4d07-42db-b2e2-ee267064e155 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 14:35:14.891245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:14.891245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 14:35:14.891245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 14:35:14.891245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 14:35:14.891245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 14:35:14.891245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 14:35:14.891245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:14.891245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 14:35:14.891245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:14.891245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 14:35:14.891245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:14.891245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:14.891245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:14.891245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:14.891245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:14.891245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:14.892304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:14.892304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:14.892304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:14.892304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:14.892304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:14.892304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:14.892304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 14:35:14.892304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 14:35:14.892304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 14:35:14.892304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 14:35:14.892304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:14.892304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:14.892304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:14.892304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:14.892304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:14.892304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:14.893155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:14.893155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:14.893155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:14.893155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:14.893155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:14.893155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:14.893155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:14.893155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:14.893155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:14.893155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:14.893155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 14:35:14.893155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:14.893155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 14:35:14.893155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:14.896345 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 438/884] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jul 20 14:35:14 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:35:14.902882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949dc2ee-e63a-4ed9-8ad6-a3f76f798d8f None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:15.011340 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949dc2ee-e63a-4ed9-8ad6-a3f76f798d8f None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:15.047061 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bfa4c8-219f-4d65-a4a1-8a3a9e3237de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:15.048469 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfa4c8-219f-4d65-a4a1-8a3a9e3237de None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:15.048951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfa4c8-219f-4d65-a4a1-8a3a9e3237de None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:15.049422 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfa4c8-219f-4d65-a4a1-8a3a9e3237de None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:15.050353 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bfa4c8-219f-4d65-a4a1-8a3a9e3237de None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:15.050854 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bfa4c8-219f-4d65-a4a1-8a3a9e3237de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:15.052474 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bfa4c8-219f-4d65-a4a1-8a3a9e3237de None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:15.058339 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 447/885] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jul 20 14:35:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:15.064361 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e3cb6-4d07-42db-b2e2-ee267064e155 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:15.169664 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e3cb6-4d07-42db-b2e2-ee267064e155 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:15.204817 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903c700-f124-45c3-9ba4-f355777043e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:15.205914 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903c700-f124-45c3-9ba4-f355777043e6 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:15.206399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903c700-f124-45c3-9ba4-f355777043e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:15.206917 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903c700-f124-45c3-9ba4-f355777043e6 None admin] PATH_INFO: `/v3/services` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:15.207897 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903c700-f124-45c3-9ba4-f355777043e6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:15.208399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903c700-f124-45c3-9ba4-f355777043e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:15.210057 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903c700-f124-45c3-9ba4-f355777043e6 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:15.216151 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 439/886] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jul 20 14:35:15 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:15.222409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bfa4c8-219f-4d65-a4a1-8a3a9e3237de None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:15.239683 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:15.327210 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bfa4c8-219f-4d65-a4a1-8a3a9e3237de None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:15.363518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87457dba-f4fa-4795-8597-93a9560b8443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:15.364195 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87457dba-f4fa-4795-8597-93a9560b8443 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:15.364482 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87457dba-f4fa-4795-8597-93a9560b8443 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:15.364762 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87457dba-f4fa-4795-8597-93a9560b8443 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:15.365654 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87457dba-f4fa-4795-8597-93a9560b8443 None admin] RBAC: Adding query filter params (interface=public, service_id=ead7275b8c0d4270aee95c967b45ba73, region_id=RegionOne) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:15.365970 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87457dba-f4fa-4795-8597-93a9560b8443 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:15.367382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87457dba-f4fa-4795-8597-93a9560b8443 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:15.373117 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 448/887] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Jul 20 14:35:15 2022] GET /identity/v3/endpoints?service_id=ead7275b8c0d4270aee95c967b45ba73&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:15.378852 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903c700-f124-45c3-9ba4-f355777043e6 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:15.403473 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:15.491158 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903c700-f124-45c3-9ba4-f355777043e6 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:15.522930 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377e72f-62b5-459f-9926-6266e37ad152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:15.523642 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377e72f-62b5-459f-9926-6266e37ad152 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:15.524045 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377e72f-62b5-459f-9926-6266e37ad152 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:15.524358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377e72f-62b5-459f-9926-6266e37ad152 None admin] PATH_INFO: `/v3/services` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:15.525052 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6377e72f-62b5-459f-9926-6266e37ad152 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:15.526449 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6377e72f-62b5-459f-9926-6266e37ad152 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:15.532728 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 440/888] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Jul 20 14:35:15 2022] GET /identity/v3/services => generated 1268 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:35:15.816756 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f7c63-6a00-4d36-ba88-13dd3d75e021 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:15.817286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f7c63-6a00-4d36-ba88-13dd3d75e021 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:15.817286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f7c63-6a00-4d36-ba88-13dd3d75e021 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:15.819021 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 449/889] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:15.824466 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b5e28-9d69-46ba-9f4e-8bc9e2905ae6 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:15.824924 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b5e28-9d69-46ba-9f4e-8bc9e2905ae6 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:15.825042 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b5e28-9d69-46ba-9f4e-8bc9e2905ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:15.843780 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-ed8b5e28-9d69-46ba-9f4e-8bc9e2905ae6 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:15.845788 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-ed8b5e28-9d69-46ba-9f4e-8bc9e2905ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:15.917245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:15.927463 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 441/890] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:15 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:35:15.949878 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f7c63-6a00-4d36-ba88-13dd3d75e021 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:16.032475 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:16.056034 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f7c63-6a00-4d36-ba88-13dd3d75e021 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:16.091081 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665b13d-7f14-4751-8a3e-c1fad231f7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:16.091839 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665b13d-7f14-4751-8a3e-c1fad231f7e3 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:16.092126 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665b13d-7f14-4751-8a3e-c1fad231f7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:16.092500 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665b13d-7f14-4751-8a3e-c1fad231f7e3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:16.093167 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6665b13d-7f14-4751-8a3e-c1fad231f7e3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:16.094554 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6665b13d-7f14-4751-8a3e-c1fad231f7e3 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:16.097394 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b563dc3-1c7a-4c1f-9976-aaa1e16a6d90 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:16.097960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b563dc3-1c7a-4c1f-9976-aaa1e16a6d90 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:16.098479 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b563dc3-1c7a-4c1f-9976-aaa1e16a6d90 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:16.100388 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 442/891] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 14:35:16.105616 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2695c2f-cc1d-40d9-9ab2-0f8e8b7a6323 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:16.106149 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2695c2f-cc1d-40d9-9ab2-0f8e8b7a6323 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:16.106633 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2695c2f-cc1d-40d9-9ab2-0f8e8b7a6323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:16.107855 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-6665b13d-7f14-4751-8a3e-c1fad231f7e3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 14:35:16.107855 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:16.107855 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 14:35:16.107855 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 14:35:16.107855 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 14:35:16.107855 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 14:35:16.107855 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 14:35:16.107855 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:16.107855 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 14:35:16.107855 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:16.107855 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 14:35:16.107855 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:16.107855 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:16.107855 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:16.107855 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:16.107855 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:16.108773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:16.108773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:16.108773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:16.108773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:16.108773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:16.108773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:16.108773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:16.108773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 14:35:16.108773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 14:35:16.108773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 14:35:16.108773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 14:35:16.108773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:16.108773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:16.108773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:16.108773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:16.108773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:16.109627 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:16.109627 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:16.109627 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:16.109627 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:16.109627 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:16.109627 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:16.109627 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:16.109627 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:16.109627 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:16.109627 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:16.109627 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:16.109627 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 14:35:16.109627 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:16.109627 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 14:35:16.109627 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:16.109627 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 450/892] 10.222.0.15 () {56 vars in 1132 bytes} [Wed Jul 20 14:35:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:16.114950 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665b13d-7f14-4751-8a3e-c1fad231f7e3 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:16.128549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-a2695c2f-cc1d-40d9-9ab2-0f8e8b7a6323 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:16.130167 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-a2695c2f-cc1d-40d9-9ab2-0f8e8b7a6323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:16.215508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 443/893] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:16 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 14:35:16.218228 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665b13d-7f14-4751-8a3e-c1fad231f7e3 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:16.233503 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2695c2f-cc1d-40d9-9ab2-0f8e8b7a6323 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:16.257932 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34cd8ab-580a-432b-a8e3-aa754ac4f91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:16.258626 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cd8ab-580a-432b-a8e3-aa754ac4f91b None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:16.258917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cd8ab-580a-432b-a8e3-aa754ac4f91b None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:16.259654 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cd8ab-580a-432b-a8e3-aa754ac4f91b None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:16.260051 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34cd8ab-580a-432b-a8e3-aa754ac4f91b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:16.260367 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34cd8ab-580a-432b-a8e3-aa754ac4f91b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:16.261895 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34cd8ab-580a-432b-a8e3-aa754ac4f91b None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:16.267570 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 451/894] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Jul 20 14:35:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:16.273117 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34cd8ab-580a-432b-a8e3-aa754ac4f91b None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:16.282344 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:16.350077 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2695c2f-cc1d-40d9-9ab2-0f8e8b7a6323 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:16.376580 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34cd8ab-580a-432b-a8e3-aa754ac4f91b None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:16.380520 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:16.391109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457bed1e-7847-49ba-ba24-ab78a2b74de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:16.392273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457bed1e-7847-49ba-ba24-ab78a2b74de7 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:16.392800 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457bed1e-7847-49ba-ba24-ab78a2b74de7 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:16.393380 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457bed1e-7847-49ba-ba24-ab78a2b74de7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:16.405434 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457bed1e-7847-49ba-ba24-ab78a2b74de7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:35:16.411376 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a3679-8bb0-42c0-a2c5-9d9f6cf1ef62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:16.412060 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a3679-8bb0-42c0-a2c5-9d9f6cf1ef62 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:16.412414 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a3679-8bb0-42c0-a2c5-9d9f6cf1ef62 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:16.412687 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a3679-8bb0-42c0-a2c5-9d9f6cf1ef62 None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:16.413432 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31a3679-8bb0-42c0-a2c5-9d9f6cf1ef62 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:16.413740 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31a3679-8bb0-42c0-a2c5-9d9f6cf1ef62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:16.415265 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31a3679-8bb0-42c0-a2c5-9d9f6cf1ef62 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:16.417131 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-457bed1e-7847-49ba-ba24-ab78a2b74de7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 14:35:16.417131 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:16.417131 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:16.417131 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:16.417131 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:16.417131 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:16.417131 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:16.417131 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:16.417131 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:16.417131 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:16.417131 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:16.417131 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:16.417131 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:35:16.417131 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:35:16.417131 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:35:16.417131 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:35:16.418044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:35:16.418044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:35:16.418044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:35:16.418044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:35:16.418044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:16.418044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:16.418044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:16.418044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:16.418044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:16.418044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:16.418044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:16.418044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:16.418044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:16.418044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:16.418044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:16.418044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:16.418968 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:16.418968 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:16.418968 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:16.418968 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:16.418968 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:35:16.418968 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:35:16.418968 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:35:16.418968 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:35:16.418968 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:35:16.418968 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:35:16.418968 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 14:35:16.418968 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:16.420762 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 452/895] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Jul 20 14:35:16 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:16.427167 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 444/896] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Jul 20 14:35:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:35:16.427294 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a3679-8bb0-42c0-a2c5-9d9f6cf1ef62 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:16.428533 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457bed1e-7847-49ba-ba24-ab78a2b74de7 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:16.499688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:16.532552 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a3679-8bb0-42c0-a2c5-9d9f6cf1ef62 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:16.544355 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457bed1e-7847-49ba-ba24-ab78a2b74de7 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:16.567803 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e6cf7-e627-42cd-9541-f80ca50102e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:16.568440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e6cf7-e627-42cd-9541-f80ca50102e6 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:16.568732 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e6cf7-e627-42cd-9541-f80ca50102e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:16.569015 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e6cf7-e627-42cd-9541-f80ca50102e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:16.569853 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e6cf7-e627-42cd-9541-f80ca50102e6 None admin] RBAC: Adding query filter params (interface=public, service_id=e5bae7d132f54cf885d75b1e1aff5027, region_id=RegionOne) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:16.570127 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e6cf7-e627-42cd-9541-f80ca50102e6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:16.571550 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e6cf7-e627-42cd-9541-f80ca50102e6 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:16.581651 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 453/897] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Jul 20 14:35:16 2022] GET /identity/v3/endpoints?service_id=e5bae7d132f54cf885d75b1e1aff5027&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:16.582536 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e6cf7-e627-42cd-9541-f80ca50102e6 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:16.584970 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87574e91-9aaa-4c77-8003-dc6b2488826f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:16.586018 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87574e91-9aaa-4c77-8003-dc6b2488826f None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:16.586691 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87574e91-9aaa-4c77-8003-dc6b2488826f None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:16.587331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87574e91-9aaa-4c77-8003-dc6b2488826f None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:16.588303 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87574e91-9aaa-4c77-8003-dc6b2488826f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:16.588813 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87574e91-9aaa-4c77-8003-dc6b2488826f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:16.590418 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87574e91-9aaa-4c77-8003-dc6b2488826f None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:16.604875 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 445/898] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Jul 20 14:35:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:16.606193 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87574e91-9aaa-4c77-8003-dc6b2488826f None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:16.663131 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:16.686711 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e6cf7-e627-42cd-9541-f80ca50102e6 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:16.724250 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf47661-6cc1-466a-b50c-c23ad32fc254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:16.724990 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf47661-6cc1-466a-b50c-c23ad32fc254 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:16.725280 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf47661-6cc1-466a-b50c-c23ad32fc254 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:16.725560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf47661-6cc1-466a-b50c-c23ad32fc254 None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:16.725876 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87574e91-9aaa-4c77-8003-dc6b2488826f None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:16.726576 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf47661-6cc1-466a-b50c-c23ad32fc254 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:16.728265 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf47661-6cc1-466a-b50c-c23ad32fc254 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:16.730104 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:16.734895 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 454/899] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Jul 20 14:35:16 2022] GET /identity/v3/services => generated 1268 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:35:16.768954 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208ff61-869f-430b-a8f4-cbf89aa4ff63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:16.770170 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208ff61-869f-430b-a8f4-cbf89aa4ff63 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:16.770685 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208ff61-869f-430b-a8f4-cbf89aa4ff63 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:16.771211 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208ff61-869f-430b-a8f4-cbf89aa4ff63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:16.784320 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b208ff61-869f-430b-a8f4-cbf89aa4ff63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:16.786208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b208ff61-869f-430b-a8f4-cbf89aa4ff63 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:16.806870 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-b208ff61-869f-430b-a8f4-cbf89aa4ff63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:16.806870 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:16.806870 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:16.806870 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:16.806870 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:16.806870 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:16.806870 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:16.806870 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:16.806870 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:16.806870 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:16.806870 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:16.806870 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:16.806870 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:35:16.806870 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:35:16.806870 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:35:16.807986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:35:16.807986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:16.807986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:16.807986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:16.807986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:16.807986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:16.807986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:16.807986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:16.807986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:16.807986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:16.807986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:16.807986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:16.807986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:16.807986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:16.807986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:16.807986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:16.807986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:16.808909 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:35:16.808909 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:35:16.808909 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:16.808909 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:16.810973 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 446/900] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:16.821457 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf47661-6cc1-466a-b50c-c23ad32fc254 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:16.840626 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9a925-c4f2-4b70-8ad1-c3d6e212e2ba None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:16.841184 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9a925-c4f2-4b70-8ad1-c3d6e212e2ba None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:16.841263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9a925-c4f2-4b70-8ad1-c3d6e212e2ba None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:16.842766 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 447/901] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 14:35:16.849709 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b056fa-6821-420f-a594-7b59014618f1 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:16.850122 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b056fa-6821-420f-a594-7b59014618f1 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:16.850414 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b056fa-6821-420f-a594-7b59014618f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:16.878740 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-a1b056fa-6821-420f-a594-7b59014618f1 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:16.880398 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-a1b056fa-6821-420f-a594-7b59014618f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:16.961519 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf47661-6cc1-466a-b50c-c23ad32fc254 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:16.970152 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 448/902] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:16 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 14:35:16.989374 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b056fa-6821-420f-a594-7b59014618f1 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:17.004683 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b29f89-ce28-42e2-9955-361d16f9c2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:17.006014 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b29f89-ce28-42e2-9955-361d16f9c2a3 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:17.006964 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b29f89-ce28-42e2-9955-361d16f9c2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:17.007471 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b29f89-ce28-42e2-9955-361d16f9c2a3 None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:17.008660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b29f89-ce28-42e2-9955-361d16f9c2a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:17.009398 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b29f89-ce28-42e2-9955-361d16f9c2a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:17.011438 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b29f89-ce28-42e2-9955-361d16f9c2a3 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:17.024158 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:17.030273 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 455/903] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:17.036050 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b29f89-ce28-42e2-9955-361d16f9c2a3 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:17.099086 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:17.104016 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b056fa-6821-420f-a594-7b59014618f1 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:17.141715 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8436c9a-c0a3-4a21-ae9a-2038cfa40309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:17.142766 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8436c9a-c0a3-4a21-ae9a-2038cfa40309 None admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:17.143291 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8436c9a-c0a3-4a21-ae9a-2038cfa40309 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:17.143485 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b29f89-ce28-42e2-9955-361d16f9c2a3 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:17.143956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8436c9a-c0a3-4a21-ae9a-2038cfa40309 None admin] PATH_INFO: `/v3/services` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:17.144993 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8436c9a-c0a3-4a21-ae9a-2038cfa40309 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:17.146712 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8436c9a-c0a3-4a21-ae9a-2038cfa40309 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:17.171272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 449/904] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Jul 20 14:35:16 2022] POST /identity/v3/services => generated 241 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 14:35:17.189792 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9cec75-369b-4c2a-89ca-db3cdfaf3a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:17.190638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9cec75-369b-4c2a-89ca-db3cdfaf3a3b None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:17.190922 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9cec75-369b-4c2a-89ca-db3cdfaf3a3b None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:17.191226 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9cec75-369b-4c2a-89ca-db3cdfaf3a3b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:17.200913 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9cec75-369b-4c2a-89ca-db3cdfaf3a3b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:17.202573 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9cec75-369b-4c2a-89ca-db3cdfaf3a3b None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:17.220510 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-dc9cec75-369b-4c2a-89ca-db3cdfaf3a3b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 14:35:17.220510 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:17.220510 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:17.220510 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:17.220510 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:17.220510 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:17.220510 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:17.220510 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:17.220510 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:17.220510 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:17.220510 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:17.220510 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:17.220510 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:35:17.220510 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:35:17.220510 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:35:17.220510 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:35:17.221517 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:17.221517 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:17.221517 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:35:17.221517 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:35:17.221517 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:35:17.221517 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:35:17.221517 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:17.221517 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:17.221517 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:17.221517 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:17.221517 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:17.221517 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:17.221517 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:17.221517 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:17.221517 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:17.221517 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:17.221517 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:17.221517 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:17.222920 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:17.222920 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:17.222920 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:35:17.222920 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:35:17.222920 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:35:17.222920 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:35:17.222920 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:35:17.222920 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:35:17.222920 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 14:35:17.222920 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:17.222920 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 456/905] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Jul 20 14:35:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:17.228324 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8436c9a-c0a3-4a21-ae9a-2038cfa40309 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:17.332880 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:17.345617 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8436c9a-c0a3-4a21-ae9a-2038cfa40309 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:17.383004 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb5e0f-b1bc-4c5d-84c5-b6606d7ec841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:17.383719 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb5e0f-b1bc-4c5d-84c5-b6606d7ec841 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:17.384536 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb5e0f-b1bc-4c5d-84c5-b6606d7ec841 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:17.384536 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb5e0f-b1bc-4c5d-84c5-b6606d7ec841 None admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:17.385181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb5e0f-b1bc-4c5d-84c5-b6606d7ec841 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:17.385486 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb5e0f-b1bc-4c5d-84c5-b6606d7ec841 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:17.386928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb5e0f-b1bc-4c5d-84c5-b6606d7ec841 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:17.406097 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 450/906] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Jul 20 14:35:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:17.411931 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9cec75-369b-4c2a-89ca-db3cdfaf3a3b None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:17.518487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9cec75-369b-4c2a-89ca-db3cdfaf3a3b None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:17.553723 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:17.554942 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d81987-881d-480c-a727-3483364c0f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:17.555679 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d81987-881d-480c-a727-3483364c0f33 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:17.556300 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d81987-881d-480c-a727-3483364c0f33 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:17.556581 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d81987-881d-480c-a727-3483364c0f33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:17.626563 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d81987-881d-480c-a727-3483364c0f33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:17.628063 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d81987-881d-480c-a727-3483364c0f33 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:17.644706 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb3844-08fb-413e-ba28-86608596008f None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:17.645011 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb3844-08fb-413e-ba28-86608596008f None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:17.645276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb3844-08fb-413e-ba28-86608596008f None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:17.647098 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 451/907] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 14:35:17.652362 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5855ed-0477-4d87-8319-3b5f4045c48b None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:17.652683 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5855ed-0477-4d87-8319-3b5f4045c48b None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:17.653063 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5855ed-0477-4d87-8319-3b5f4045c48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:17.670524 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-af5855ed-0477-4d87-8319-3b5f4045c48b None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:17.671804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-af5855ed-0477-4d87-8319-3b5f4045c48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:17.729474 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 457/908] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:35:17 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 14:35:17.736838 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d81987-881d-480c-a727-3483364c0f33 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:17.755137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 452/909] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 14:35:17.759425 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:17.773678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5855ed-0477-4d87-8319-3b5f4045c48b None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:17.857014 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d81987-881d-480c-a727-3483364c0f33 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:17.869912 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:17.893366 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5855ed-0477-4d87-8319-3b5f4045c48b None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:17.906329 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cadeebf-01f2-485a-bce0-226384cf49ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:17.908149 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cadeebf-01f2-485a-bce0-226384cf49ad None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:17.908583 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cadeebf-01f2-485a-bce0-226384cf49ad None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:17.909005 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cadeebf-01f2-485a-bce0-226384cf49ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:17.924515 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cadeebf-01f2-485a-bce0-226384cf49ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:17.926068 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cadeebf-01f2-485a-bce0-226384cf49ad None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:17.936116 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f9009-421a-4675-8e14-7cf7cca6ec14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:17.937648 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f9009-421a-4675-8e14-7cf7cca6ec14 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:17.938353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f9009-421a-4675-8e14-7cf7cca6ec14 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:17.938775 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f9009-421a-4675-8e14-7cf7cca6ec14 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:17.939641 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940f9009-421a-4675-8e14-7cf7cca6ec14 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:17.941165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940f9009-421a-4675-8e14-7cf7cca6ec14 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:17.946724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-9cadeebf-01f2-485a-bce0-226384cf49ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:17.946724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:17.946724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:17.946724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:17.946724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:17.946724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:17.946724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:17.946724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:17.946724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:17.946724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:17.946724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:17.946724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:17.946724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:35:17.946724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:35:17.946724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:35:17.947822 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:35:17.947822 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:17.947822 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:17.947822 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:17.947822 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:17.947822 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:17.947822 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:17.947822 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:17.947822 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:17.947822 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:17.947822 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:17.947822 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:17.947822 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:17.947822 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:17.947822 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:17.947822 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:17.947822 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:17.948955 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:35:17.948955 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:35:17.948955 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:17.948955 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:17.954814 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 458/910] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:17.956157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cadeebf-01f2-485a-bce0-226384cf49ad None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:17.956928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-940f9009-421a-4675-8e14-7cf7cca6ec14 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 14:35:17.956928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:17.956928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 14:35:17.956928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 14:35:17.956928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 14:35:17.956928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 14:35:17.956928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 14:35:17.956928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:17.956928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 14:35:17.956928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:17.956928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 14:35:17.956928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:17.956928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:17.956928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:17.956928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:17.956928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:17.956928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:17.958060 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:17.958060 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:17.958060 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:17.958060 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:17.958060 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:17.958060 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:17.958060 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 14:35:17.958060 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 14:35:17.958060 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 14:35:17.958060 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 14:35:17.958060 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:17.958060 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:17.958060 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:17.958060 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:17.958060 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:17.958060 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:17.959159 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:17.959159 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:17.959159 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:17.959159 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:17.959159 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:17.959159 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:17.959159 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:17.959159 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:17.959159 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:17.959159 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:17.959159 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 14:35:17.959159 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:17.959159 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 14:35:17.959159 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:17.962103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 453/911] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jul 20 14:35:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:17.968959 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f9009-421a-4675-8e14-7cf7cca6ec14 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:18.032724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:18.070207 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cadeebf-01f2-485a-bce0-226384cf49ad None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:18.105361 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09c7a5-0306-4334-8eea-f606beab8ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:18.106225 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09c7a5-0306-4334-8eea-f606beab8ad6 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:18.106601 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09c7a5-0306-4334-8eea-f606beab8ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:18.106968 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09c7a5-0306-4334-8eea-f606beab8ad6 None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:18.107828 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f9009-421a-4675-8e14-7cf7cca6ec14 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:18.107951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b09c7a5-0306-4334-8eea-f606beab8ad6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:18.108324 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b09c7a5-0306-4334-8eea-f606beab8ad6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:18.109790 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b09c7a5-0306-4334-8eea-f606beab8ad6 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:18.126474 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 459/912] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:35:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:18.127958 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09c7a5-0306-4334-8eea-f606beab8ad6 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:18.149869 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a243b1e-513c-49f8-b497-24e84aed356b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:18.151353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a243b1e-513c-49f8-b497-24e84aed356b None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:18.151784 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a243b1e-513c-49f8-b497-24e84aed356b None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:18.152334 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a243b1e-513c-49f8-b497-24e84aed356b None admin] PATH_INFO: `/v3/services` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:18.153327 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a243b1e-513c-49f8-b497-24e84aed356b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:18.154028 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a243b1e-513c-49f8-b497-24e84aed356b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:18.156331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a243b1e-513c-49f8-b497-24e84aed356b None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:18.163824 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 454/913] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jul 20 14:35:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:18.169590 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a243b1e-513c-49f8-b497-24e84aed356b None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:18.181921 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:18.244447 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09c7a5-0306-4334-8eea-f606beab8ad6 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:18.278471 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:18.291324 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bebbd13-0b01-4a3c-811d-0c68b7fd624a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:18.292386 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bebbd13-0b01-4a3c-811d-0c68b7fd624a None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:18.292658 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a243b1e-513c-49f8-b497-24e84aed356b None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:18.292971 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bebbd13-0b01-4a3c-811d-0c68b7fd624a None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:18.293397 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bebbd13-0b01-4a3c-811d-0c68b7fd624a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:18.303303 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bebbd13-0b01-4a3c-811d-0c68b7fd624a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:18.305521 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bebbd13-0b01-4a3c-811d-0c68b7fd624a None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:18.324557 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-6bebbd13-0b01-4a3c-811d-0c68b7fd624a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:35:18.324557 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:18.324557 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:18.324557 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:18.324557 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:18.324557 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:18.324557 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:18.324557 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:18.324557 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:18.324557 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:18.324557 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:18.324557 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:18.324557 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:35:18.324557 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:35:18.324557 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:35:18.325631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:35:18.325631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:18.325631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:18.325631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:18.325631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:18.325631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:18.325631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:18.325631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:18.325631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:18.325631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:18.325631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:18.325631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:18.325631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:18.325631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:18.325631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:18.325631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:18.325631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:18.326601 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:35:18.326601 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:35:18.326601 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:35:18.326601 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:35:18.326601 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:35:18.326601 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:35:18.326601 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:35:18.326601 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:18.328979 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 460/914] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:35:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:18.335505 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bebbd13-0b01-4a3c-811d-0c68b7fd624a None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:18.339354 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b51f97-01b2-47f2-9ae5-24a5cf03befd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:18.340532 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b51f97-01b2-47f2-9ae5-24a5cf03befd None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:18.340532 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b51f97-01b2-47f2-9ae5-24a5cf03befd None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:18.340703 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b51f97-01b2-47f2-9ae5-24a5cf03befd None admin] PATH_INFO: `/v3/services` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:18.341416 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b51f97-01b2-47f2-9ae5-24a5cf03befd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:18.341793 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b51f97-01b2-47f2-9ae5-24a5cf03befd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:18.343266 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b51f97-01b2-47f2-9ae5-24a5cf03befd None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:18.353312 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 455/915] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jul 20 14:35:18 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:18.354749 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b51f97-01b2-47f2-9ae5-24a5cf03befd None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:18.431316 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:18.470756 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b51f97-01b2-47f2-9ae5-24a5cf03befd None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:18.492950 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bebbd13-0b01-4a3c-811d-0c68b7fd624a None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:18.508849 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ef929-97c2-4c73-afad-452b82a2af70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:18.509617 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ef929-97c2-4c73-afad-452b82a2af70 None admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:18.510030 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ef929-97c2-4c73-afad-452b82a2af70 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:18.510377 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ef929-97c2-4c73-afad-452b82a2af70 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:18.511267 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50ef929-97c2-4c73-afad-452b82a2af70 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:18.512841 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50ef929-97c2-4c73-afad-452b82a2af70 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:18.543078 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 456/916] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Jul 20 14:35:18 2022] POST /identity/v3/endpoints => generated 346 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:35:18.556222 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246cc582-123d-43c0-b856-971a572dc046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:18.556966 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246cc582-123d-43c0-b856-971a572dc046 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:18.557249 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246cc582-123d-43c0-b856-971a572dc046 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:18.557522 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246cc582-123d-43c0-b856-971a572dc046 None admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:18.558401 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246cc582-123d-43c0-b856-971a572dc046 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:18.558718 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246cc582-123d-43c0-b856-971a572dc046 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:18.560323 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246cc582-123d-43c0-b856-971a572dc046 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:18.574881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 461/917] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:35:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:18.580947 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ef929-97c2-4c73-afad-452b82a2af70 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:18.698307 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ef929-97c2-4c73-afad-452b82a2af70 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:18.727530 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:18.738074 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb53b8c-0a87-40d8-b153-6f02b96dcbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:18.739230 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb53b8c-0a87-40d8-b153-6f02b96dcbe0 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:18.739903 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb53b8c-0a87-40d8-b153-6f02b96dcbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:18.740207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb53b8c-0a87-40d8-b153-6f02b96dcbe0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:18.741113 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb53b8c-0a87-40d8-b153-6f02b96dcbe0 None admin] RBAC: Adding query filter params (role.id=59c1c6b5011a4faea5f02d1961352d67, scope.project.id=59333e4f593346e184877d6bd9d22026, user.id=2f8cf6d908314281a3ddc9d9513d39c2) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:18.741425 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb53b8c-0a87-40d8-b153-6f02b96dcbe0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:18.742889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb53b8c-0a87-40d8-b153-6f02b96dcbe0 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:18.755193 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 457/918] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:35:18 2022] GET /identity/v3/role_assignments?user.id=2f8cf6d908314281a3ddc9d9513d39c2&scope.project.id=59333e4f593346e184877d6bd9d22026&role.id=59c1c6b5011a4faea5f02d1961352d67 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:18.837789 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02fcfc-ef53-4f28-b6d7-f6c15660c38d None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:18.838162 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02fcfc-ef53-4f28-b6d7-f6c15660c38d None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:18.838334 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02fcfc-ef53-4f28-b6d7-f6c15660c38d None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:18.840223 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 462/919] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:18.845983 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc8cb4-5eb4-469f-a82e-ca9655b83585 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:18.846301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc8cb4-5eb4-469f-a82e-ca9655b83585 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:18.846548 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc8cb4-5eb4-469f-a82e-ca9655b83585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:18.865099 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-a8fc8cb4-5eb4-469f-a82e-ca9655b83585 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:18.866472 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-a8fc8cb4-5eb4-469f-a82e-ca9655b83585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:18.943086 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 458/920] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:35:18.966478 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c02fcfc-ef53-4f28-b6d7-f6c15660c38d None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:19.018900 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:19.079506 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c02fcfc-ef53-4f28-b6d7-f6c15660c38d None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:19.115951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba3b71-ba69-4beb-8a8e-c4fedea72088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:19.116785 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba3b71-ba69-4beb-8a8e-c4fedea72088 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:19.117070 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba3b71-ba69-4beb-8a8e-c4fedea72088 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:19.117349 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba3b71-ba69-4beb-8a8e-c4fedea72088 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:19.118080 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ba3b71-ba69-4beb-8a8e-c4fedea72088 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:19.119596 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ba3b71-ba69-4beb-8a8e-c4fedea72088 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:19.132492 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-c9ba3b71-ba69-4beb-8a8e-c4fedea72088 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 14:35:19.132492 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:19.132492 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 14:35:19.132492 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 14:35:19.132492 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 14:35:19.132492 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 14:35:19.132492 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 14:35:19.132492 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:19.132492 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 14:35:19.132492 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:19.132492 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 14:35:19.132492 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:19.132492 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:19.132492 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:19.132492 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:19.132492 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:19.133564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:19.133564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:19.133564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:19.133564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:19.133564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:19.133564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:19.133564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:19.133564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 14:35:19.133564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 14:35:19.133564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 14:35:19.133564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 14:35:19.133564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:19.133564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:19.133564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:19.133564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:19.133564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:19.134500 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:19.134500 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:19.134500 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:19.134500 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:19.134500 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:19.134500 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:19.134500 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:19.134500 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:19.134500 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:19.134500 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:19.134500 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:19.134500 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 14:35:19.134500 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:19.134500 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 14:35:19.134500 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:19.134500 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 463/921] 10.222.0.15 () {56 vars in 1132 bytes} [Wed Jul 20 14:35:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:19.142517 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc8cb4-5eb4-469f-a82e-ca9655b83585 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:19.249281 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc8cb4-5eb4-469f-a82e-ca9655b83585 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:19.273877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:19.286587 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba6834d-bff8-45f8-90b4-9fef85ef3add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:19.287748 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba6834d-bff8-45f8-90b4-9fef85ef3add None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:19.288259 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba6834d-bff8-45f8-90b4-9fef85ef3add None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:19.288746 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba6834d-bff8-45f8-90b4-9fef85ef3add None admin] PATH_INFO: `/v3/services` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:19.289667 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba6834d-bff8-45f8-90b4-9fef85ef3add None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:19.290183 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba6834d-bff8-45f8-90b4-9fef85ef3add None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:19.291814 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba6834d-bff8-45f8-90b4-9fef85ef3add None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:19.298235 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 459/922] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Jul 20 14:35:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:19.304517 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba3b71-ba69-4beb-8a8e-c4fedea72088 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:19.376981 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9307a0-8fa2-4ab9-878d-b970e1b028d5 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:19.377778 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9307a0-8fa2-4ab9-878d-b970e1b028d5 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:19.378450 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9307a0-8fa2-4ab9-878d-b970e1b028d5 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:19.381222 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 460/923] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:19.387580 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd971c51-e41f-46f0-b7aa-3d9814c4decc None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:19.388194 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd971c51-e41f-46f0-b7aa-3d9814c4decc None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:19.388691 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd971c51-e41f-46f0-b7aa-3d9814c4decc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:19.408573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-dd971c51-e41f-46f0-b7aa-3d9814c4decc None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:19.409776 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-dd971c51-e41f-46f0-b7aa-3d9814c4decc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:19.421984 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba3b71-ba69-4beb-8a8e-c4fedea72088 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:19.439534 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:19.466138 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434e49d-e8a1-474f-b5b2-6e69f878d8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:19.467201 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434e49d-e8a1-474f-b5b2-6e69f878d8cd None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:19.467759 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434e49d-e8a1-474f-b5b2-6e69f878d8cd None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:19.468283 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434e49d-e8a1-474f-b5b2-6e69f878d8cd None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:19.469274 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434e49d-e8a1-474f-b5b2-6e69f878d8cd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:19.469745 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434e49d-e8a1-474f-b5b2-6e69f878d8cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:19.471470 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434e49d-e8a1-474f-b5b2-6e69f878d8cd None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:19.482654 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 464/924] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Jul 20 14:35:19 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:19.484268 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434e49d-e8a1-474f-b5b2-6e69f878d8cd None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:19.503075 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 461/925] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 14:35:19.522377 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd971c51-e41f-46f0-b7aa-3d9814c4decc None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:19.599720 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434e49d-e8a1-474f-b5b2-6e69f878d8cd None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:19.638096 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e78ca7-b896-44ee-affa-7bb351e9762f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:19.639074 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e78ca7-b896-44ee-affa-7bb351e9762f None admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:19.639507 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e78ca7-b896-44ee-affa-7bb351e9762f None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:19.639920 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e78ca7-b896-44ee-affa-7bb351e9762f None admin] PATH_INFO: `/v3/endpoints` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:19.640824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e78ca7-b896-44ee-affa-7bb351e9762f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:19.642512 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e78ca7-b896-44ee-affa-7bb351e9762f None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:19.650604 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd971c51-e41f-46f0-b7aa-3d9814c4decc None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:19.675612 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 465/926] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Jul 20 14:35:19 2022] POST /identity/v3/endpoints => generated 347 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:35:19.680320 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:19.708557 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc0154-7cb5-4d6a-9298-a708ec1956bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:19.709608 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc0154-7cb5-4d6a-9298-a708ec1956bb None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:19.709939 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc0154-7cb5-4d6a-9298-a708ec1956bb None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:19.710261 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc0154-7cb5-4d6a-9298-a708ec1956bb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:19.710939 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc0154-7cb5-4d6a-9298-a708ec1956bb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:19.712395 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc0154-7cb5-4d6a-9298-a708ec1956bb None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:19.726949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-b8fc0154-7cb5-4d6a-9298-a708ec1956bb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 14:35:19.726949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:19.726949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 14:35:19.726949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 14:35:19.726949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 14:35:19.726949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 14:35:19.726949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 14:35:19.726949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:19.726949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 14:35:19.726949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:19.726949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 14:35:19.726949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:19.726949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:19.726949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:19.726949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:19.726949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:19.726949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:19.728845 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:19.728845 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:19.728845 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:19.728845 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:19.728845 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:19.728845 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:19.728845 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 14:35:19.728845 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 14:35:19.728845 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 14:35:19.728845 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 14:35:19.728845 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:19.728845 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:19.728845 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:19.728845 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:19.728845 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:19.728845 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:19.730004 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:19.730004 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:19.730004 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:19.730004 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:19.730004 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:19.730004 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:19.730004 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:19.730004 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:19.730004 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:19.730004 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:19.730004 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 14:35:19.730004 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:19.730004 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 14:35:19.730004 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:19.730004 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 462/927] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Jul 20 14:35:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:35:19.737916 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e78ca7-b896-44ee-affa-7bb351e9762f None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:19.769606 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:19.855179 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e78ca7-b896-44ee-affa-7bb351e9762f None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:19.891330 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867703c9-f293-4dd9-b11a-d67796588c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:19.892412 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867703c9-f293-4dd9-b11a-d67796588c83 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:19.892901 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867703c9-f293-4dd9-b11a-d67796588c83 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:19.893389 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867703c9-f293-4dd9-b11a-d67796588c83 None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:19.894321 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867703c9-f293-4dd9-b11a-d67796588c83 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:19.894790 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867703c9-f293-4dd9-b11a-d67796588c83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:19.896390 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867703c9-f293-4dd9-b11a-d67796588c83 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:19.902121 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 466/928] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Jul 20 14:35:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:19.908344 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc0154-7cb5-4d6a-9298-a708ec1956bb None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:20.017280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc0154-7cb5-4d6a-9298-a708ec1956bb None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:20.052619 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589118c9-fede-46c7-bf56-4ee54cf2303a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:20.053613 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589118c9-fede-46c7-bf56-4ee54cf2303a None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:20.054058 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589118c9-fede-46c7-bf56-4ee54cf2303a None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:20.054488 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589118c9-fede-46c7-bf56-4ee54cf2303a None admin] PATH_INFO: `/v3/services` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:20.055386 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589118c9-fede-46c7-bf56-4ee54cf2303a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:20.055855 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589118c9-fede-46c7-bf56-4ee54cf2303a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:20.057428 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589118c9-fede-46c7-bf56-4ee54cf2303a None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:20.062952 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 463/929] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Jul 20 14:35:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:20.072269 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867703c9-f293-4dd9-b11a-d67796588c83 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:20.167118 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:20.177372 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867703c9-f293-4dd9-b11a-d67796588c83 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:20.214273 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454bfe22-c0c3-4491-9c00-5eb916f48b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:20.215004 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454bfe22-c0c3-4491-9c00-5eb916f48b6e None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:20.215322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454bfe22-c0c3-4491-9c00-5eb916f48b6e None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:20.215603 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454bfe22-c0c3-4491-9c00-5eb916f48b6e None admin] PATH_INFO: `/v3/endpoints` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:20.216458 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454bfe22-c0c3-4491-9c00-5eb916f48b6e None admin] RBAC: Adding query filter params (interface=public, service_id=403abedeb5d0401197b5b6312433894f, region_id=RegionOne) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:20.216781 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454bfe22-c0c3-4491-9c00-5eb916f48b6e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:20.218178 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454bfe22-c0c3-4491-9c00-5eb916f48b6e None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:20.224199 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 467/930] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Jul 20 14:35:20 2022] GET /identity/v3/endpoints?service_id=403abedeb5d0401197b5b6312433894f&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:20.230240 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589118c9-fede-46c7-bf56-4ee54cf2303a None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:20.275088 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:20.338630 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589118c9-fede-46c7-bf56-4ee54cf2303a None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:20.373584 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7800e-e1e0-4802-ab76-ed8382aa9ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:20.374259 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7800e-e1e0-4802-ab76-ed8382aa9ca1 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:20.374541 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7800e-e1e0-4802-ab76-ed8382aa9ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:20.374820 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7800e-e1e0-4802-ab76-ed8382aa9ca1 None admin] PATH_INFO: `/v3/services` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:20.375536 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c7800e-e1e0-4802-ab76-ed8382aa9ca1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:20.376906 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c7800e-e1e0-4802-ab76-ed8382aa9ca1 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:20.383454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 464/931] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Jul 20 14:35:20 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:35:20.746023 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39780b6-7f7a-4540-8c15-eeeebe9686a2 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:20.746218 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39780b6-7f7a-4540-8c15-eeeebe9686a2 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:20.746508 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39780b6-7f7a-4540-8c15-eeeebe9686a2 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:20.748157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 468/932] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:20.753553 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca294a33-8a80-48b1-8b38-36139b243585 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:20.753843 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca294a33-8a80-48b1-8b38-36139b243585 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:20.754121 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca294a33-8a80-48b1-8b38-36139b243585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:20.774965 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-ca294a33-8a80-48b1-8b38-36139b243585 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:20.776214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-ca294a33-8a80-48b1-8b38-36139b243585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:20.851007 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3b462-9b2e-4fed-adec-f5841a7a26b3 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:20.851377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3b462-9b2e-4fed-adec-f5841a7a26b3 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:20.851597 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3b462-9b2e-4fed-adec-f5841a7a26b3 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:20.854171 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 469/933] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 14:35:20.858279 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f83a0-32d7-4f8c-98fb-d072a9b296ff None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:20.858565 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f83a0-32d7-4f8c-98fb-d072a9b296ff None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:20.858838 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f83a0-32d7-4f8c-98fb-d072a9b296ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:20.869173 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 465/934] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:35:20.878445 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-1c5f83a0-32d7-4f8c-98fb-d072a9b296ff None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:20.879666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-1c5f83a0-32d7-4f8c-98fb-d072a9b296ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:20.894141 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca294a33-8a80-48b1-8b38-36139b243585 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:20.929550 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:20.967672 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 470/935] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:35:20.990676 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f83a0-32d7-4f8c-98fb-d072a9b296ff None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:21.008055 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca294a33-8a80-48b1-8b38-36139b243585 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:21.044846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe283f9-5bf5-4500-b724-c5da3b9c4206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:21.045517 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe283f9-5bf5-4500-b724-c5da3b9c4206 None admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:21.045795 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe283f9-5bf5-4500-b724-c5da3b9c4206 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:21.046070 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe283f9-5bf5-4500-b724-c5da3b9c4206 None admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:21.046781 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe283f9-5bf5-4500-b724-c5da3b9c4206 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:21.048620 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe283f9-5bf5-4500-b724-c5da3b9c4206 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:21.050009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: WARNING keystone.server.flask.common [None req-dfe283f9-5bf5-4500-b724-c5da3b9c4206 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 20 14:35:21.070132 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.sql.core [None req-dfe283f9-5bf5-4500-b724-c5da3b9c4206 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 14:35:21.070132 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 20 14:35:21.070132 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [parameters: {'id': '67b4a5c2185b4ccdb00ef1c082dbbf65', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 20 14:35:21.070132 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86861) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 14:35:21.086949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-dfe283f9-5bf5-4500-b724-c5da3b9c4206 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 67b4a5c2185b4ccdb00ef1c082dbbf65.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 67b4a5c2185b4ccdb00ef1c082dbbf65. Jul 20 14:35:21.086949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:21.086949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 14:35:21.086949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 14:35:21.086949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 14:35:21.086949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 14:35:21.086949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 14:35:21.086949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 14:35:21.086949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 14:35:21.086949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application conn.query(q) Jul 20 14:35:21.086949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 14:35:21.086949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 14:35:21.086949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 14:35:21.087889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application result.read() Jul 20 14:35:21.087889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 14:35:21.087889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 14:35:21.087889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 14:35:21.087889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 14:35:21.087889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 14:35:21.087889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 14:35:21.087889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 14:35:21.087889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 14:35:21.087889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 14:35:21.087889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:21.087889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 14:35:21.087889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:21.087889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:21.087889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 14:35:21.087889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 14:35:21.087889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 20 14:35:21.087889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 20 14:35:21.089294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 14:35:21.089294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application next(self.gen) Jul 20 14:35:21.089294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 14:35:21.089294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application yield resource Jul 20 14:35:21.089294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 14:35:21.089294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application next(self.gen) Jul 20 14:35:21.089294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 14:35:21.089294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 14:35:21.089294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 14:35:21.089294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 14:35:21.089294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 14:35:21.089294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise self.value Jul 20 14:35:21.089294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 14:35:21.089294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 14:35:21.089294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 14:35:21.089294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application session.commit() Jul 20 14:35:21.089294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 14:35:21.090423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 14:35:21.090423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 14:35:21.090423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 14:35:21.090423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 14:35:21.090423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self.session.flush() Jul 20 14:35:21.090423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 14:35:21.090423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 14:35:21.090423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 14:35:21.090423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 14:35:21.090423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 14:35:21.090423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application compat.raise_( Jul 20 14:35:21.090423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 14:35:21.090423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception Jul 20 14:35:21.090423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 14:35:21.090423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 14:35:21.090423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 14:35:21.090423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 14:35:21.091706 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 14:35:21.091706 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 14:35:21.091706 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 14:35:21.091706 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 14:35:21.091706 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 20 14:35:21.091706 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 20 14:35:21.091706 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 14:35:21.091706 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 14:35:21.091706 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 14:35:21.091706 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 14:35:21.091706 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 14:35:21.091706 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 14:35:21.091706 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 14:35:21.091706 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 14:35:21.091706 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 14:35:21.092485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 14:35:21.092485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 14:35:21.092485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception Jul 20 14:35:21.092485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 14:35:21.092485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 14:35:21.092485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 14:35:21.092485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 14:35:21.092485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 14:35:21.092485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 14:35:21.092485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 14:35:21.092485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application conn.query(q) Jul 20 14:35:21.092485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 14:35:21.092485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 14:35:21.092485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 14:35:21.092485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application result.read() Jul 20 14:35:21.092485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 14:35:21.092485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 14:35:21.094238 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 14:35:21.094238 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 14:35:21.094238 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 14:35:21.094238 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 14:35:21.094238 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 14:35:21.094238 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 14:35:21.094238 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 14:35:21.094238 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 20 14:35:21.094238 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [parameters: {'id': '67b4a5c2185b4ccdb00ef1c082dbbf65', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 20 14:35:21.094238 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 14:35:21.094238 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:21.094238 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 14:35:21.094238 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:21.094238 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:21.094238 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 20 14:35:21.096097 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 20 14:35:21.096097 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 14:35:21.096097 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 14:35:21.096097 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 20 14:35:21.096097 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:21.096097 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 14:35:21.096097 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:21.096097 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:21.096097 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:21.096097 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:21.096097 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:21.096097 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:21.096097 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:21.096097 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:21.096097 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:21.096097 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:21.096097 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:21.098488 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:21.098488 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 20 14:35:21.098488 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 20 14:35:21.098488 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:21.098488 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:21.098488 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 20 14:35:21.098488 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 20 14:35:21.098488 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 67b4a5c2185b4ccdb00ef1c082dbbf65. Jul 20 14:35:21.098488 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:21.098488 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 466/936] 10.222.0.15 () {60 vars in 1140 bytes} [Wed Jul 20 14:35:20 2022] POST /identity/v3/projects => generated 262 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:35:21.098488 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe283f9-5bf5-4500-b724-c5da3b9c4206 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:21.115120 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:21.139312 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f83a0-32d7-4f8c-98fb-d072a9b296ff None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:21.186123 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a625ac1-0a61-4f7a-a92f-c7897985940b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:21.187160 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a625ac1-0a61-4f7a-a92f-c7897985940b None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:21.187537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a625ac1-0a61-4f7a-a92f-c7897985940b None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:21.187977 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a625ac1-0a61-4f7a-a92f-c7897985940b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:21.198590 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a625ac1-0a61-4f7a-a92f-c7897985940b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:35:21.199813 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:21.210217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-2a625ac1-0a61-4f7a-a92f-c7897985940b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 14:35:21.210217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:21.210217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:21.210217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:21.210217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:21.210217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:21.210217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:21.210217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:21.210217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:21.210217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:21.210217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:21.210217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:21.210217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:35:21.210217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:35:21.210217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:35:21.210217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:35:21.211664 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:35:21.211664 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:35:21.211664 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:35:21.211664 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:35:21.211664 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:21.211664 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:21.211664 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:21.211664 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:21.211664 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:21.211664 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:21.211664 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:21.211664 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:21.211664 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:21.211664 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:21.211664 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:21.211664 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:21.212679 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:21.212679 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:21.212679 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:21.212679 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:21.212679 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:35:21.212679 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:35:21.212679 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:35:21.212679 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:35:21.212679 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:35:21.212679 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:35:21.212679 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 14:35:21.212679 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:21.219508 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 471/937] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Jul 20 14:35:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:35:21.220932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe283f9-5bf5-4500-b724-c5da3b9c4206 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:21.223256 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a625ac1-0a61-4f7a-a92f-c7897985940b None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:21.262792 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9701a43b-7d9e-489c-83c2-2e16b3223300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:21.264077 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701a43b-7d9e-489c-83c2-2e16b3223300 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:21.264576 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701a43b-7d9e-489c-83c2-2e16b3223300 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:21.265035 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701a43b-7d9e-489c-83c2-2e16b3223300 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:21.272482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9701a43b-7d9e-489c-83c2-2e16b3223300 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:21.274508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9701a43b-7d9e-489c-83c2-2e16b3223300 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:21.289937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-9701a43b-7d9e-489c-83c2-2e16b3223300 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:35:21.289937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:21.289937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:21.289937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:21.289937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:21.289937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:21.289937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:21.289937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:21.289937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:21.289937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:21.289937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:21.289937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:21.289937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:35:21.289937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:35:21.289937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:35:21.290839 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:35:21.290839 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:21.290839 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:21.290839 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:21.290839 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:21.290839 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:21.290839 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:21.290839 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:21.290839 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:21.290839 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:21.290839 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:21.290839 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:21.290839 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:21.290839 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:21.290839 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:21.290839 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:21.290839 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:21.291753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:35:21.291753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:35:21.291753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:35:21.291753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:35:21.291753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:35:21.291753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:35:21.291753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:35:21.291753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:21.298829 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 467/938] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:35:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:21.300279 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9701a43b-7d9e-489c-83c2-2e16b3223300 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:21.348084 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a625ac1-0a61-4f7a-a92f-c7897985940b None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:21.384831 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438e5649-3e6b-474f-b9e7-a0c52ffb93e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:21.385553 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e5649-3e6b-474f-b9e7-a0c52ffb93e9 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:21.386248 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e5649-3e6b-474f-b9e7-a0c52ffb93e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:21.386533 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e5649-3e6b-474f-b9e7-a0c52ffb93e9 None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:21.387335 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438e5649-3e6b-474f-b9e7-a0c52ffb93e9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:21.387649 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438e5649-3e6b-474f-b9e7-a0c52ffb93e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:21.389416 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438e5649-3e6b-474f-b9e7-a0c52ffb93e9 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:21.400841 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 472/939] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Jul 20 14:35:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:21.406505 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438e5649-3e6b-474f-b9e7-a0c52ffb93e9 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:21.417860 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9701a43b-7d9e-489c-83c2-2e16b3223300 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:21.418525 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:21.463223 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888cba0a-4c17-4ac0-89c6-b7a82b22ca58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:21.463223 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888cba0a-4c17-4ac0-89c6-b7a82b22ca58 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:21.463223 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888cba0a-4c17-4ac0-89c6-b7a82b22ca58 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:21.463223 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888cba0a-4c17-4ac0-89c6-b7a82b22ca58 None admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:21.463941 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888cba0a-4c17-4ac0-89c6-b7a82b22ca58 None admin] RBAC: Adding query filter params (name=service) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:21.464255 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888cba0a-4c17-4ac0-89c6-b7a82b22ca58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:21.466017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888cba0a-4c17-4ac0-89c6-b7a82b22ca58 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:21.493630 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 468/940] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Jul 20 14:35:21 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:21.528887 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438e5649-3e6b-474f-b9e7-a0c52ffb93e9 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:21.569833 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508d1ac-03a3-4e52-bc6a-c13b8ec57618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:21.570668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508d1ac-03a3-4e52-bc6a-c13b8ec57618 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:21.570954 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508d1ac-03a3-4e52-bc6a-c13b8ec57618 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:21.571316 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508d1ac-03a3-4e52-bc6a-c13b8ec57618 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:21.584918 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8508d1ac-03a3-4e52-bc6a-c13b8ec57618 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:21.586419 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8508d1ac-03a3-4e52-bc6a-c13b8ec57618 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:21.605661 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-8508d1ac-03a3-4e52-bc6a-c13b8ec57618 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:21.605661 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:21.605661 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:21.605661 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:21.605661 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:21.605661 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:21.605661 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:21.605661 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:21.605661 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:21.605661 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:21.605661 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:21.605661 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:21.605661 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:35:21.605661 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:35:21.605661 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:35:21.606616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:35:21.606616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:21.606616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:21.606616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:21.606616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:21.606616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:21.606616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:21.606616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:21.606616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:21.606616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:21.606616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:21.606616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:21.606616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:21.606616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:21.606616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:21.606616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:21.606616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:21.607962 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:35:21.607962 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:35:21.607962 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:21.607962 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:21.607962 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 473/941] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:21.614574 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888cba0a-4c17-4ac0-89c6-b7a82b22ca58 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:21.628405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:21.725263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888cba0a-4c17-4ac0-89c6-b7a82b22ca58 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:21.761173 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf31e6-9396-4d5f-9704-865cee4cf88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:21.761852 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf31e6-9396-4d5f-9704-865cee4cf88c None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:21.762134 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf31e6-9396-4d5f-9704-865cee4cf88c None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:21.762408 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf31e6-9396-4d5f-9704-865cee4cf88c None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:21.763155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faf31e6-9396-4d5f-9704-865cee4cf88c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:21.763464 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faf31e6-9396-4d5f-9704-865cee4cf88c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:21.764943 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faf31e6-9396-4d5f-9704-865cee4cf88c None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:21.777619 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 469/942] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:21.783751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508d1ac-03a3-4e52-bc6a-c13b8ec57618 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:21.846228 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:21.889016 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508d1ac-03a3-4e52-bc6a-c13b8ec57618 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:21.923858 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cfd937-692b-4693-b8bf-e1efc9f60983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:21.924691 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfd937-692b-4693-b8bf-e1efc9f60983 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:21.924979 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfd937-692b-4693-b8bf-e1efc9f60983 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:21.925252 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfd937-692b-4693-b8bf-e1efc9f60983 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:21.933363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cfd937-692b-4693-b8bf-e1efc9f60983 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:21.934823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cfd937-692b-4693-b8bf-e1efc9f60983 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:21.941293 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3799df4-b220-4440-ba63-deae65935b6a None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:21.941841 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3799df4-b220-4440-ba63-deae65935b6a None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:21.942327 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3799df4-b220-4440-ba63-deae65935b6a None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:21.944260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 470/943] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 14:35:21.949068 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00837210-060b-4aa1-b071-7a3705b3c321 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:21.949334 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00837210-060b-4aa1-b071-7a3705b3c321 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:21.949580 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00837210-060b-4aa1-b071-7a3705b3c321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:21.949968 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-48cfd937-692b-4693-b8bf-e1efc9f60983 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 14:35:21.949968 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:21.949968 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:21.949968 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:21.949968 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:21.949968 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:21.949968 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:21.949968 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:21.949968 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:21.949968 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:21.949968 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:21.949968 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:21.949968 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:35:21.949968 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:35:21.949968 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:35:21.949968 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:35:21.950885 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:21.950885 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:21.950885 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:35:21.950885 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:35:21.950885 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:35:21.950885 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:35:21.950885 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:21.950885 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:21.950885 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:21.950885 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:21.950885 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:21.950885 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:21.950885 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:21.950885 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:21.950885 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:21.950885 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:21.950885 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:21.950885 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:21.952284 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:21.952284 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:21.952284 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:35:21.952284 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:35:21.952284 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:35:21.952284 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:35:21.952284 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:35:21.952284 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:35:21.952284 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 14:35:21.952284 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:21.952284 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 474/944] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Jul 20 14:35:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:21.960212 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cfd937-692b-4693-b8bf-e1efc9f60983 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:21.970683 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-00837210-060b-4aa1-b071-7a3705b3c321 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:21.972703 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-00837210-060b-4aa1-b071-7a3705b3c321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:22.042325 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:22.049042 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 471/945] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 14:35:22.070556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00837210-060b-4aa1-b071-7a3705b3c321 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:22.080159 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cfd937-692b-4693-b8bf-e1efc9f60983 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:22.102061 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:22.126603 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd311a4-f2f6-481a-b89d-1dcae6fdf5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:22.128036 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd311a4-f2f6-481a-b89d-1dcae6fdf5ba None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:22.128477 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd311a4-f2f6-481a-b89d-1dcae6fdf5ba None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:22.128923 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd311a4-f2f6-481a-b89d-1dcae6fdf5ba None admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:22.129918 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd311a4-f2f6-481a-b89d-1dcae6fdf5ba None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:22.130382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd311a4-f2f6-481a-b89d-1dcae6fdf5ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:22.132115 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd311a4-f2f6-481a-b89d-1dcae6fdf5ba None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:22.151862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 475/946] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Jul 20 14:35:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:22.163425 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd311a4-f2f6-481a-b89d-1dcae6fdf5ba None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:22.193591 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00837210-060b-4aa1-b071-7a3705b3c321 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:22.235718 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92815ece-6a15-4626-91fe-e3dd118238a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:22.237004 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92815ece-6a15-4626-91fe-e3dd118238a6 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:22.237699 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92815ece-6a15-4626-91fe-e3dd118238a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:22.238465 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92815ece-6a15-4626-91fe-e3dd118238a6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:22.239655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92815ece-6a15-4626-91fe-e3dd118238a6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:22.241701 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92815ece-6a15-4626-91fe-e3dd118238a6 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:22.255533 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-92815ece-6a15-4626-91fe-e3dd118238a6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 14:35:22.255533 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:22.255533 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 14:35:22.255533 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 14:35:22.255533 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 14:35:22.255533 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 14:35:22.255533 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 14:35:22.255533 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:22.255533 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 14:35:22.255533 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:22.255533 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 14:35:22.255533 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:22.255533 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:22.255533 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:22.255533 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:22.255533 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:22.255533 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:22.256877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:22.256877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:22.256877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:22.256877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:22.256877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:22.256877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:22.256877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 14:35:22.256877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 14:35:22.256877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 14:35:22.256877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 14:35:22.256877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:22.256877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:22.256877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:22.256877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:22.256877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:22.256877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:22.257932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:22.257932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:22.257932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:22.257932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:22.257932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:22.257932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:22.257932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:22.257932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:22.257932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:22.257932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:22.257932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 14:35:22.257932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:22.257932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 14:35:22.257932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:22.260628 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 472/947] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:35:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:35:22.266664 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92815ece-6a15-4626-91fe-e3dd118238a6 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:22.274409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd311a4-f2f6-481a-b89d-1dcae6fdf5ba None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:22.310826 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd8b7fb-c4b6-4296-9dcb-7a361f24fce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:22.312051 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8b7fb-c4b6-4296-9dcb-7a361f24fce2 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:22.312528 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8b7fb-c4b6-4296-9dcb-7a361f24fce2 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:22.312996 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8b7fb-c4b6-4296-9dcb-7a361f24fce2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:22.323992 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd8b7fb-c4b6-4296-9dcb-7a361f24fce2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:22.325748 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd8b7fb-c4b6-4296-9dcb-7a361f24fce2 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:22.342212 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:22.344371 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-3cd8b7fb-c4b6-4296-9dcb-7a361f24fce2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:22.344371 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:22.344371 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:22.344371 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:22.344371 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:22.344371 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:22.344371 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:22.344371 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:22.344371 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:22.344371 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:22.344371 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:22.344371 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:22.344371 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:35:22.344371 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:35:22.344371 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:35:22.346259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:35:22.346259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:22.346259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:22.346259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:22.346259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:22.346259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:22.346259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:22.346259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:22.346259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:22.346259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:22.346259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:22.346259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:22.346259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:22.346259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:22.346259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:22.346259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:22.346259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:22.347224 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:35:22.347224 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:35:22.347224 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:22.347224 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:22.353790 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 476/948] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:22.355088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd8b7fb-c4b6-4296-9dcb-7a361f24fce2 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:22.383893 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:22.387380 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92815ece-6a15-4626-91fe-e3dd118238a6 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:22.426888 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c0f7c5-1e7c-4d0f-9918-dd5fcde282ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:22.427695 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c0f7c5-1e7c-4d0f-9918-dd5fcde282ca None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:22.427909 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c0f7c5-1e7c-4d0f-9918-dd5fcde282ca None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:22.428194 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c0f7c5-1e7c-4d0f-9918-dd5fcde282ca None admin] PATH_INFO: `/v3/services` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:22.428919 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c0f7c5-1e7c-4d0f-9918-dd5fcde282ca None admin] RBAC: Adding query filter params (name=compute) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:22.429221 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c0f7c5-1e7c-4d0f-9918-dd5fcde282ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:22.430727 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c0f7c5-1e7c-4d0f-9918-dd5fcde282ca None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:22.441158 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 473/949] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Jul 20 14:35:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:22.441775 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c0f7c5-1e7c-4d0f-9918-dd5fcde282ca None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:22.473009 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd8b7fb-c4b6-4296-9dcb-7a361f24fce2 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:22.522998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8600327e-c980-411f-a7b1-e798737497a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:22.523709 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600327e-c980-411f-a7b1-e798737497a9 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:22.523996 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600327e-c980-411f-a7b1-e798737497a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:22.524387 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600327e-c980-411f-a7b1-e798737497a9 None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:22.525240 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8600327e-c980-411f-a7b1-e798737497a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:22.525582 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8600327e-c980-411f-a7b1-e798737497a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:22.527072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8600327e-c980-411f-a7b1-e798737497a9 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:22.544700 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 477/950] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:35:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:22.544700 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac776dc3-254b-4549-8ef0-b8aed476be4b None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:22.544700 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac776dc3-254b-4549-8ef0-b8aed476be4b None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:22.544700 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac776dc3-254b-4549-8ef0-b8aed476be4b None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:22.544700 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 478/951] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:22.548231 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac776dc3-254b-4549-8ef0-b8aed476be4b None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:22.564201 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c0f7c5-1e7c-4d0f-9918-dd5fcde282ca None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:22.592426 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:22.600762 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e65f3-7b23-4ad2-b083-c69e97c90850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:22.601503 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e65f3-7b23-4ad2-b083-c69e97c90850 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:22.601853 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e65f3-7b23-4ad2-b083-c69e97c90850 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:22.602199 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e65f3-7b23-4ad2-b083-c69e97c90850 None admin] PATH_INFO: `/v3/services` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:22.602988 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e65f3-7b23-4ad2-b083-c69e97c90850 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:22.603386 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e65f3-7b23-4ad2-b083-c69e97c90850 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:22.604832 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e65f3-7b23-4ad2-b083-c69e97c90850 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:22.609950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 474/952] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Jul 20 14:35:22 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:22.611244 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70242672-d7d0-48e2-9eae-7225dc392ef4 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:22.611585 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70242672-d7d0-48e2-9eae-7225dc392ef4 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:22.611925 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70242672-d7d0-48e2-9eae-7225dc392ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:22.629866 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-70242672-d7d0-48e2-9eae-7225dc392ef4 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:22.631468 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-70242672-d7d0-48e2-9eae-7225dc392ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:22.662123 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:22.673343 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac776dc3-254b-4549-8ef0-b8aed476be4b None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:22.719812 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf57180-57f5-4bcf-addc-afbf28c2ff6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:22.721321 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf57180-57f5-4bcf-addc-afbf28c2ff6b None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:22.721612 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf57180-57f5-4bcf-addc-afbf28c2ff6b None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:22.721893 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf57180-57f5-4bcf-addc-afbf28c2ff6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:22.731718 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 475/953] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:35:22.736744 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf57180-57f5-4bcf-addc-afbf28c2ff6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:22.738417 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf57180-57f5-4bcf-addc-afbf28c2ff6b None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:22.747309 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70242672-d7d0-48e2-9eae-7225dc392ef4 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:22.755841 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-3bf57180-57f5-4bcf-addc-afbf28c2ff6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:35:22.755841 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:22.755841 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:22.755841 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:22.755841 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:22.755841 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:22.755841 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:22.755841 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:22.755841 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:22.755841 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:22.755841 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:22.755841 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:22.755841 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:35:22.755841 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:35:22.755841 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:35:22.756921 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:35:22.756921 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:22.756921 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:22.756921 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:22.756921 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:22.756921 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:22.756921 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:22.756921 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:22.756921 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:22.756921 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:22.756921 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:22.756921 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:22.756921 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:22.756921 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:22.756921 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:22.756921 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:22.756921 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:22.758166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:35:22.758166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:35:22.758166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:35:22.758166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:35:22.758166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:35:22.758166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:35:22.758166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:35:22.758166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:22.758166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 479/954] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:35:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:22.763558 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf57180-57f5-4bcf-addc-afbf28c2ff6b None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:22.881886 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70242672-d7d0-48e2-9eae-7225dc392ef4 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:22.884761 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf57180-57f5-4bcf-addc-afbf28c2ff6b None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:22.911440 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:22.925576 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb930fb3-b2b8-4864-a0d5-86e757f92ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:22.926841 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb930fb3-b2b8-4864-a0d5-86e757f92ff6 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:22.927520 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb930fb3-b2b8-4864-a0d5-86e757f92ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:22.927970 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb930fb3-b2b8-4864-a0d5-86e757f92ff6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:22.928204 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793709d0-2743-403d-899f-98f55e8296d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:22.928976 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793709d0-2743-403d-899f-98f55e8296d7 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:22.929151 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb930fb3-b2b8-4864-a0d5-86e757f92ff6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:22.929446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793709d0-2743-403d-899f-98f55e8296d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:22.929823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793709d0-2743-403d-899f-98f55e8296d7 None admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:22.930726 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793709d0-2743-403d-899f-98f55e8296d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:22.931197 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793709d0-2743-403d-899f-98f55e8296d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:22.931356 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb930fb3-b2b8-4864-a0d5-86e757f92ff6 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:22.932809 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793709d0-2743-403d-899f-98f55e8296d7 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:22.944280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-fb930fb3-b2b8-4864-a0d5-86e757f92ff6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 14:35:22.944280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:22.944280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 14:35:22.944280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 14:35:22.944280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 14:35:22.944280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 14:35:22.944280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 14:35:22.944280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:22.944280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 14:35:22.944280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:22.944280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 14:35:22.944280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:22.944280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:22.944280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:22.944280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:22.944280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:22.944280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:22.945590 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:22.945590 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:22.945590 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:22.945590 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:22.945590 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:22.945590 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:22.945590 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 14:35:22.945590 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 14:35:22.945590 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 14:35:22.945590 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 14:35:22.945590 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:22.945590 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:22.945590 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:22.945590 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:22.945590 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:22.945590 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:22.946752 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:22.946752 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:22.946752 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:22.946752 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:22.946752 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:22.946752 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:22.946752 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:22.946752 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:22.946752 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:22.946752 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:22.946752 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 14:35:22.946752 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:22.946752 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 14:35:22.946752 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:22.949288 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 476/955] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Jul 20 14:35:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:35:22.951194 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 480/956] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:35:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:22.959661 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb930fb3-b2b8-4864-a0d5-86e757f92ff6 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:22.960157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793709d0-2743-403d-899f-98f55e8296d7 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:22.971635 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:23.064278 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb930fb3-b2b8-4864-a0d5-86e757f92ff6 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:23.072886 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793709d0-2743-403d-899f-98f55e8296d7 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:23.098200 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24698c31-e9c6-4d5e-92af-2870395dc756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:23.098885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24698c31-e9c6-4d5e-92af-2870395dc756 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:23.099199 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24698c31-e9c6-4d5e-92af-2870395dc756 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:23.099619 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24698c31-e9c6-4d5e-92af-2870395dc756 None admin] PATH_INFO: `/v3/services` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:23.100338 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24698c31-e9c6-4d5e-92af-2870395dc756 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:23.100649 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24698c31-e9c6-4d5e-92af-2870395dc756 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:23.102027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24698c31-e9c6-4d5e-92af-2870395dc756 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:23.107561 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 477/957] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Jul 20 14:35:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:23.113035 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24698c31-e9c6-4d5e-92af-2870395dc756 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:23.113926 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bad092-5ea3-47e1-8ba1-5ceef4599805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:23.115124 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bad092-5ea3-47e1-8ba1-5ceef4599805 None admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:23.115747 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bad092-5ea3-47e1-8ba1-5ceef4599805 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:23.116225 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bad092-5ea3-47e1-8ba1-5ceef4599805 None admin] PATH_INFO: `/v3/projects/59333e4f593346e184877d6bd9d22026/users/2f8cf6d908314281a3ddc9d9513d39c2/roles/59c1c6b5011a4faea5f02d1961352d67` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:23.118841 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bad092-5ea3-47e1-8ba1-5ceef4599805 None admin] RBAC: Authorizing `identity:create_grant(project_id=59333e4f593346e184877d6bd9d22026, user_id=2f8cf6d908314281a3ddc9d9513d39c2, role_id=59c1c6b5011a4faea5f02d1961352d67)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:23.120536 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bad092-5ea3-47e1-8ba1-5ceef4599805 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:23.147168 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 481/958] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jul 20 14:35:22 2022] PUT /identity/v3/projects/59333e4f593346e184877d6bd9d22026/users/2f8cf6d908314281a3ddc9d9513d39c2/roles/59c1c6b5011a4faea5f02d1961352d67 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:35:23.183098 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:23.239216 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24698c31-e9c6-4d5e-92af-2870395dc756 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:23.280767 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98525d73-d2b5-483a-966f-60c3c71fa66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:23.281553 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98525d73-d2b5-483a-966f-60c3c71fa66d None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:23.281902 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98525d73-d2b5-483a-966f-60c3c71fa66d None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:23.282244 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98525d73-d2b5-483a-966f-60c3c71fa66d None admin] PATH_INFO: `/v3/services` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:23.283032 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98525d73-d2b5-483a-966f-60c3c71fa66d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:23.283437 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98525d73-d2b5-483a-966f-60c3c71fa66d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:23.284910 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98525d73-d2b5-483a-966f-60c3c71fa66d None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:23.291474 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 478/959] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Jul 20 14:35:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:23.297988 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bad092-5ea3-47e1-8ba1-5ceef4599805 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:23.378551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:23.423683 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bad092-5ea3-47e1-8ba1-5ceef4599805 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:23.462031 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dcaf6c-54a1-40f1-a693-8ac2716cc2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:23.462699 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dcaf6c-54a1-40f1-a693-8ac2716cc2d8 None admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:23.463030 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dcaf6c-54a1-40f1-a693-8ac2716cc2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:23.463337 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dcaf6c-54a1-40f1-a693-8ac2716cc2d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:23.464091 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dcaf6c-54a1-40f1-a693-8ac2716cc2d8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:23.465462 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dcaf6c-54a1-40f1-a693-8ac2716cc2d8 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:23.487422 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 482/960] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Jul 20 14:35:23 2022] POST /identity/v3/endpoints => generated 349 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:35:23.753594 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8025e-5f5f-4796-a797-796acbf0cc4b None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:23.753955 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8025e-5f5f-4796-a797-796acbf0cc4b None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:23.754322 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8025e-5f5f-4796-a797-796acbf0cc4b None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:23.756152 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 479/961] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:23.762116 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357a495-7170-40a1-8423-972c38c0485c None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:23.762493 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357a495-7170-40a1-8423-972c38c0485c None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:23.762844 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357a495-7170-40a1-8423-972c38c0485c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:23.781370 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-3357a495-7170-40a1-8423-972c38c0485c None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:23.782585 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-3357a495-7170-40a1-8423-972c38c0485c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:23.865458 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 483/962] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:35:23.890551 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8025e-5f5f-4796-a797-796acbf0cc4b None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:24.001102 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8025e-5f5f-4796-a797-796acbf0cc4b None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:24.015922 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:24.036634 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d110c39-77e6-4406-9f9d-3bbad40f365a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:24.037387 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d110c39-77e6-4406-9f9d-3bbad40f365a None admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:24.037668 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d110c39-77e6-4406-9f9d-3bbad40f365a None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:24.037945 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d110c39-77e6-4406-9f9d-3bbad40f365a None admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:24.038665 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d110c39-77e6-4406-9f9d-3bbad40f365a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:24.040175 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d110c39-77e6-4406-9f9d-3bbad40f365a None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:24.054372 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.sql.core [None req-8d110c39-77e6-4406-9f9d-3bbad40f365a None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 14:35:24.054372 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 20 14:35:24.054372 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [parameters: {'user_id': '06f6a0489fb048c7a437e1a4822cea1b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 20 14:35:24.054372 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86861) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 14:35:24.062114 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-8d110c39-77e6-4406-9f9d-3bbad40f365a None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 20 14:35:24.062114 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:24.062114 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 14:35:24.062114 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 14:35:24.062114 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 14:35:24.062114 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 14:35:24.062114 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 14:35:24.062114 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 14:35:24.062114 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 14:35:24.062114 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application conn.query(q) Jul 20 14:35:24.062114 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 14:35:24.062114 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 14:35:24.062114 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 14:35:24.062114 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application result.read() Jul 20 14:35:24.062114 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 14:35:24.063220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 14:35:24.063220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 14:35:24.063220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 14:35:24.063220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 14:35:24.063220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 14:35:24.063220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 14:35:24.063220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 14:35:24.063220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 14:35:24.063220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:24.063220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 14:35:24.063220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:24.063220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:24.063220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 14:35:24.063220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 14:35:24.063220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 20 14:35:24.063220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 20 14:35:24.063220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 14:35:24.063220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application next(self.gen) Jul 20 14:35:24.064875 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 14:35:24.064875 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application yield resource Jul 20 14:35:24.064875 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 14:35:24.064875 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application next(self.gen) Jul 20 14:35:24.064875 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 14:35:24.064875 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 14:35:24.064875 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 14:35:24.064875 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 14:35:24.064875 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 14:35:24.064875 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise self.value Jul 20 14:35:24.064875 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 14:35:24.064875 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 14:35:24.064875 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 14:35:24.064875 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application session.commit() Jul 20 14:35:24.064875 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 14:35:24.064875 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 14:35:24.064875 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 14:35:24.066093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 14:35:24.066093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 14:35:24.066093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self.session.flush() Jul 20 14:35:24.066093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 14:35:24.066093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 14:35:24.066093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 14:35:24.066093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 14:35:24.066093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 14:35:24.066093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application compat.raise_( Jul 20 14:35:24.066093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 14:35:24.066093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception Jul 20 14:35:24.066093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 14:35:24.066093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 14:35:24.066093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 14:35:24.066093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 14:35:24.066093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 14:35:24.066093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 14:35:24.067407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 14:35:24.067407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 14:35:24.067407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 20 14:35:24.067407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 20 14:35:24.067407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 14:35:24.067407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 14:35:24.067407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 14:35:24.067407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 14:35:24.067407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 14:35:24.067407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 14:35:24.067407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 14:35:24.067407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 14:35:24.067407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 14:35:24.067407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 14:35:24.067407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 14:35:24.067407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception Jul 20 14:35:24.068893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 14:35:24.068893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 14:35:24.068893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 14:35:24.068893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 14:35:24.068893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 14:35:24.068893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 14:35:24.068893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 14:35:24.068893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application conn.query(q) Jul 20 14:35:24.068893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 14:35:24.068893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 14:35:24.068893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 14:35:24.068893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application result.read() Jul 20 14:35:24.068893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 14:35:24.068893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 14:35:24.068893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 14:35:24.068893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 14:35:24.069974 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 14:35:24.069974 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 14:35:24.069974 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 14:35:24.069974 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 14:35:24.069974 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 14:35:24.069974 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 20 14:35:24.069974 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [parameters: {'user_id': '06f6a0489fb048c7a437e1a4822cea1b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 20 14:35:24.069974 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 14:35:24.069974 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:24.069974 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 14:35:24.069974 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:24.069974 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:24.069974 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:24.069974 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:24.069974 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:24.075771 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:24.075771 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:24.075771 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:24.075771 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:24.075771 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:24.075771 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:24.075771 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:24.075771 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 20 14:35:24.075771 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 20 14:35:24.075771 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:24.075771 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:24.075771 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:35:24.075771 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:35:24.075771 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:35:24.075771 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:35:24.075771 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 20 14:35:24.075771 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 20 14:35:24.077940 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 20 14:35:24.077940 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 20 14:35:24.077940 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 14:35:24.077940 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 14:35:24.077940 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 20 14:35:24.077940 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:24.077940 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 480/963] 10.222.0.15 () {60 vars in 1131 bytes} [Wed Jul 20 14:35:23 2022] POST /identity/v3/users => generated 160 bytes in 174 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:24.082908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3357a495-7170-40a1-8423-972c38c0485c None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:24.130100 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:24.181899 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3357a495-7170-40a1-8423-972c38c0485c None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:24.219725 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d1d5c-54a1-4387-932c-de369a2be355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:24.220936 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d1d5c-54a1-4387-932c-de369a2be355 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:24.221513 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d1d5c-54a1-4387-932c-de369a2be355 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:24.221991 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d1d5c-54a1-4387-932c-de369a2be355 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:24.229765 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d1d5c-54a1-4387-932c-de369a2be355 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:24.231565 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d1d5c-54a1-4387-932c-de369a2be355 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:24.246521 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-e96d1d5c-54a1-4387-932c-de369a2be355 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 14:35:24.246521 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:24.246521 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:24.246521 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:24.246521 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:24.246521 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:24.246521 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:24.246521 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:24.246521 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:24.246521 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:24.246521 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:24.246521 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:24.246521 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:35:24.246521 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:35:24.246521 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:35:24.246521 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:35:24.247646 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:24.247646 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:24.247646 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:35:24.247646 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:35:24.247646 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:35:24.247646 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:35:24.247646 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:24.247646 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:24.247646 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:24.247646 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:24.247646 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:24.247646 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:24.247646 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:24.247646 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:24.247646 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:24.247646 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:24.247646 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:24.247646 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:24.248865 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:24.248865 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:24.248865 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:35:24.248865 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:35:24.248865 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:35:24.248865 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:35:24.248865 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:35:24.248865 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:35:24.248865 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 14:35:24.248865 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:24.251205 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 484/964] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:35:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:24.260579 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d110c39-77e6-4406-9f9d-3bbad40f365a None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:24.362320 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d110c39-77e6-4406-9f9d-3bbad40f365a None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:24.398405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d9618e-3a24-425f-b2a1-464588c0ab1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:24.399165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9618e-3a24-425f-b2a1-464588c0ab1b None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:24.399448 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9618e-3a24-425f-b2a1-464588c0ab1b None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:24.399721 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9618e-3a24-425f-b2a1-464588c0ab1b None admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:24.400600 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d9618e-3a24-425f-b2a1-464588c0ab1b None admin] RBAC: Adding query filter params (name=cinder) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:24.400942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d9618e-3a24-425f-b2a1-464588c0ab1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:24.402685 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d9618e-3a24-425f-b2a1-464588c0ab1b None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:24.447214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 481/965] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jul 20 14:35:24 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:24.807487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62616de5-271c-4826-afb8-2fe02118561f None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:24.807739 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62616de5-271c-4826-afb8-2fe02118561f None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:24.808042 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62616de5-271c-4826-afb8-2fe02118561f None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:24.809532 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 485/966] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:24.815373 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1abb2f-3d3f-4083-8ff1-152e9ce0f191 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:24.815910 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1abb2f-3d3f-4083-8ff1-152e9ce0f191 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:24.816383 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1abb2f-3d3f-4083-8ff1-152e9ce0f191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:24.835241 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-9d1abb2f-3d3f-4083-8ff1-152e9ce0f191 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:24.837324 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-9d1abb2f-3d3f-4083-8ff1-152e9ce0f191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:24.868254 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:24.915093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 482/967] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:35:24.935759 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62616de5-271c-4826-afb8-2fe02118561f None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:25.041870 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62616de5-271c-4826-afb8-2fe02118561f None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:25.053364 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:25.078752 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd6cd3-0de5-418c-85b4-e173aeb5fc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:25.079380 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd6cd3-0de5-418c-85b4-e173aeb5fc25 None admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:25.079669 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd6cd3-0de5-418c-85b4-e173aeb5fc25 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:25.079948 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd6cd3-0de5-418c-85b4-e173aeb5fc25 None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:25.080682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbd6cd3-0de5-418c-85b4-e173aeb5fc25 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:25.082074 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbd6cd3-0de5-418c-85b4-e173aeb5fc25 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:25.103099 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 486/968] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Jul 20 14:35:24 2022] POST /identity/v3/services => generated 242 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:35:25.345982 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a3364-4db4-4594-b45b-5689155c7375 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:25.346509 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a3364-4db4-4594-b45b-5689155c7375 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:25.346509 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a3364-4db4-4594-b45b-5689155c7375 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:25.348017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 483/969] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:25.353230 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06759f8-1b70-4f61-92b9-251a06d9ae33 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:25.353617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06759f8-1b70-4f61-92b9-251a06d9ae33 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:25.353787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06759f8-1b70-4f61-92b9-251a06d9ae33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:25.370984 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-a06759f8-1b70-4f61-92b9-251a06d9ae33 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:25.372330 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-a06759f8-1b70-4f61-92b9-251a06d9ae33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:25.450852 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 487/970] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:25 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:35:25.474562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208a3364-4db4-4594-b45b-5689155c7375 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:25.588635 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208a3364-4db4-4594-b45b-5689155c7375 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:25.627143 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087557b-9fac-4eff-b95d-9133a796c96a None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:25.627351 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087557b-9fac-4eff-b95d-9133a796c96a None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:25.627622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087557b-9fac-4eff-b95d-9133a796c96a None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:25.629026 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 488/971] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:25.634103 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38be68-7768-4563-9f12-f74985109dda None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:25.634363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38be68-7768-4563-9f12-f74985109dda None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:25.634649 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38be68-7768-4563-9f12-f74985109dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:25.641627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352ffe4-cb85-48ce-9690-cd723831f234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:25.642453 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352ffe4-cb85-48ce-9690-cd723831f234 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:25.642737 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352ffe4-cb85-48ce-9690-cd723831f234 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:25.642988 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352ffe4-cb85-48ce-9690-cd723831f234 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:25.651765 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-bc38be68-7768-4563-9f12-f74985109dda None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:25.652972 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-bc38be68-7768-4563-9f12-f74985109dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:25.653996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1352ffe4-cb85-48ce-9690-cd723831f234 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:35:25.662248 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-1352ffe4-cb85-48ce-9690-cd723831f234 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 14:35:25.662248 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:25.662248 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:25.662248 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:25.662248 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:25.662248 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:25.662248 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:25.662248 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:25.662248 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:25.662248 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:25.662248 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:25.662248 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:25.662248 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:35:25.662248 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:35:25.662248 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:35:25.662248 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:35:25.663232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:35:25.663232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:35:25.663232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:35:25.663232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:35:25.663232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:25.663232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:25.663232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:25.663232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:25.663232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:25.663232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:25.663232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:25.663232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:25.663232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:25.663232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:25.663232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:25.663232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:25.664078 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:25.664078 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:25.664078 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:25.664078 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:25.664078 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:35:25.664078 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:35:25.664078 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:35:25.664078 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:35:25.664078 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:35:25.664078 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:35:25.664078 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 14:35:25.664078 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:25.664078 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 484/972] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Jul 20 14:35:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:25.669345 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352ffe4-cb85-48ce-9690-cd723831f234 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:25.697638 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:25.730896 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 489/973] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:25 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 14:35:25.753055 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38be68-7768-4563-9f12-f74985109dda None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:25.781484 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:25.781680 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352ffe4-cb85-48ce-9690-cd723831f234 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:25.819488 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53271ce-08d9-4202-af5a-3fa7f30934eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:25.820245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53271ce-08d9-4202-af5a-3fa7f30934eb None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:25.820463 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53271ce-08d9-4202-af5a-3fa7f30934eb None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:25.820797 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53271ce-08d9-4202-af5a-3fa7f30934eb None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:25.821563 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53271ce-08d9-4202-af5a-3fa7f30934eb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:25.821876 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53271ce-08d9-4202-af5a-3fa7f30934eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:25.823364 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53271ce-08d9-4202-af5a-3fa7f30934eb None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:25.833582 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 485/974] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Jul 20 14:35:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:25.839117 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53271ce-08d9-4202-af5a-3fa7f30934eb None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:25.871035 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38be68-7768-4563-9f12-f74985109dda None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:25.908117 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41fc92a-02e7-4d64-b5cc-be9959e4fb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:25.908864 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fc92a-02e7-4d64-b5cc-be9959e4fb0c None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:25.909154 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fc92a-02e7-4d64-b5cc-be9959e4fb0c None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:25.909435 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fc92a-02e7-4d64-b5cc-be9959e4fb0c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:25.910093 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41fc92a-02e7-4d64-b5cc-be9959e4fb0c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:25.911520 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41fc92a-02e7-4d64-b5cc-be9959e4fb0c None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:25.924015 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-a41fc92a-02e7-4d64-b5cc-be9959e4fb0c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 14:35:25.924015 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:25.924015 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 14:35:25.924015 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 14:35:25.924015 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 14:35:25.924015 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 14:35:25.924015 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 14:35:25.924015 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:25.924015 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 14:35:25.924015 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:25.924015 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 14:35:25.924015 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:25.924015 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:25.924015 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:25.924015 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:25.924015 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:25.924015 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:25.925047 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:25.925047 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:25.925047 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:25.925047 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:25.925047 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:25.925047 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:25.925047 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 14:35:25.925047 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 14:35:25.925047 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 14:35:25.925047 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 14:35:25.925047 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:25.925047 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:25.925047 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:25.925047 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:25.925047 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:25.925047 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:25.925908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:25.925908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:25.925908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:25.925908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:25.925908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:25.925908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:25.925908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:25.925908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:25.925908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:25.925908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:25.925908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 14:35:25.925908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:25.925908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 14:35:25.925908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:25.925908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 490/975] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Jul 20 14:35:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:35:25.931404 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41fc92a-02e7-4d64-b5cc-be9959e4fb0c None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:25.934227 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:25.952768 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53271ce-08d9-4202-af5a-3fa7f30934eb None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:25.988496 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f337b-cc47-4613-9283-7ec6f3055bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:25.989315 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f337b-cc47-4613-9283-7ec6f3055bff None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:25.989598 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f337b-cc47-4613-9283-7ec6f3055bff None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:25.989875 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f337b-cc47-4613-9283-7ec6f3055bff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:26.001019 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f337b-cc47-4613-9283-7ec6f3055bff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:26.002485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f337b-cc47-4613-9283-7ec6f3055bff None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:26.020069 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-ef8f337b-cc47-4613-9283-7ec6f3055bff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:26.020069 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:26.020069 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:26.020069 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:26.020069 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:26.020069 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:26.020069 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:26.020069 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:26.020069 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:26.020069 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:26.020069 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:26.020069 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:26.020069 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:35:26.020069 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:35:26.020069 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:35:26.021009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:35:26.021009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:26.021009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:26.021009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:26.021009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:26.021009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:26.021009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:26.021009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:26.021009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:26.021009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:26.021009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:26.021009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:26.021009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:26.021009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:26.021009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:26.021009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:26.021009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:26.026976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:35:26.026976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:35:26.026976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:26.026976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:26.026976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 486/976] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:35:26.026976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f337b-cc47-4613-9283-7ec6f3055bff None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:26.031785 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:26.043115 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41fc92a-02e7-4d64-b5cc-be9959e4fb0c None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:26.079487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89986429-a5da-40bf-92a2-5a7db9c4c664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:26.080168 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89986429-a5da-40bf-92a2-5a7db9c4c664 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:26.080457 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89986429-a5da-40bf-92a2-5a7db9c4c664 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:26.080739 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89986429-a5da-40bf-92a2-5a7db9c4c664 None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:26.081453 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89986429-a5da-40bf-92a2-5a7db9c4c664 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:26.081764 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89986429-a5da-40bf-92a2-5a7db9c4c664 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:26.083264 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89986429-a5da-40bf-92a2-5a7db9c4c664 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:26.088602 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 491/977] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Jul 20 14:35:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:26.094077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89986429-a5da-40bf-92a2-5a7db9c4c664 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:26.131603 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f337b-cc47-4613-9283-7ec6f3055bff None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:26.166378 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e567f7-9b2f-43a8-8606-7c291f9fe48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:26.166947 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e567f7-9b2f-43a8-8606-7c291f9fe48f None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:26.167262 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e567f7-9b2f-43a8-8606-7c291f9fe48f None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:26.167542 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e567f7-9b2f-43a8-8606-7c291f9fe48f None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:26.168270 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e567f7-9b2f-43a8-8606-7c291f9fe48f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:26.168580 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e567f7-9b2f-43a8-8606-7c291f9fe48f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:26.169969 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e567f7-9b2f-43a8-8606-7c291f9fe48f None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:26.178878 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:26.187220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 487/978] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:35:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:26.192341 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e567f7-9b2f-43a8-8606-7c291f9fe48f None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:26.211359 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89986429-a5da-40bf-92a2-5a7db9c4c664 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:26.247444 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc6fdb0-2be7-48d3-9209-e6f44a2d7a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:26.248154 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6fdb0-2be7-48d3-9209-e6f44a2d7a2b None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:26.248442 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6fdb0-2be7-48d3-9209-e6f44a2d7a2b None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:26.248719 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6fdb0-2be7-48d3-9209-e6f44a2d7a2b None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:26.249452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc6fdb0-2be7-48d3-9209-e6f44a2d7a2b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:26.249764 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc6fdb0-2be7-48d3-9209-e6f44a2d7a2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:26.251222 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc6fdb0-2be7-48d3-9209-e6f44a2d7a2b None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:26.261660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 492/979] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Jul 20 14:35:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:26.263073 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc6fdb0-2be7-48d3-9209-e6f44a2d7a2b None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:26.284337 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:26.310675 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e567f7-9b2f-43a8-8606-7c291f9fe48f None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:26.347105 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe3acdf-d80a-409c-ad58-682c18a91898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:26.347939 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3acdf-d80a-409c-ad58-682c18a91898 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:26.348221 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3acdf-d80a-409c-ad58-682c18a91898 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:26.348495 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3acdf-d80a-409c-ad58-682c18a91898 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:26.355831 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe3acdf-d80a-409c-ad58-682c18a91898 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:26.357280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe3acdf-d80a-409c-ad58-682c18a91898 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:26.367326 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc6fdb0-2be7-48d3-9209-e6f44a2d7a2b None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:26.372821 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-ffe3acdf-d80a-409c-ad58-682c18a91898 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 14:35:26.372821 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:26.372821 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:26.372821 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:26.372821 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:26.372821 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:26.372821 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:26.372821 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:26.372821 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:26.372821 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:26.372821 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:26.372821 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:26.372821 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:35:26.372821 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:35:26.372821 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:35:26.372821 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:35:26.373976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:26.373976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:26.373976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:35:26.373976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:35:26.373976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:35:26.373976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:35:26.373976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:26.373976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:26.373976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:26.373976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:26.373976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:26.373976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:26.373976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:26.373976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:26.373976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:26.373976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:26.373976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:26.373976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:26.375777 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:26.375777 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:26.375777 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:35:26.375777 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:35:26.375777 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:35:26.375777 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:35:26.375777 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:35:26.375777 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:35:26.375777 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 14:35:26.375777 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:26.378887 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 488/980] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Jul 20 14:35:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:26.379805 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe3acdf-d80a-409c-ad58-682c18a91898 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:26.405650 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de1428-c08e-4baf-8b31-0ca0aea82efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:26.406326 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de1428-c08e-4baf-8b31-0ca0aea82efb None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:26.406617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de1428-c08e-4baf-8b31-0ca0aea82efb None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:26.406896 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de1428-c08e-4baf-8b31-0ca0aea82efb None admin] PATH_INFO: `/v3/endpoints` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:26.407746 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de1428-c08e-4baf-8b31-0ca0aea82efb None admin] RBAC: Adding query filter params (interface=admin, service_id=403abedeb5d0401197b5b6312433894f, region_id=RegionOne) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:26.408062 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de1428-c08e-4baf-8b31-0ca0aea82efb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:26.409492 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de1428-c08e-4baf-8b31-0ca0aea82efb None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:26.414739 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 493/981] 10.222.0.15 () {56 vars in 1249 bytes} [Wed Jul 20 14:35:26 2022] GET /identity/v3/endpoints?service_id=403abedeb5d0401197b5b6312433894f&interface=admin®ion_id=RegionOne => generated 193 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:26.421081 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de1428-c08e-4baf-8b31-0ca0aea82efb None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:26.441050 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:26.490369 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe3acdf-d80a-409c-ad58-682c18a91898 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:26.513028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:26.525551 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e599b-3d9f-4389-8ea3-d1c554193914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:26.526327 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e599b-3d9f-4389-8ea3-d1c554193914 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:26.526536 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e599b-3d9f-4389-8ea3-d1c554193914 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:26.526778 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e599b-3d9f-4389-8ea3-d1c554193914 None admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:26.527651 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e599b-3d9f-4389-8ea3-d1c554193914 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:26.527967 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e599b-3d9f-4389-8ea3-d1c554193914 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:26.529402 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e599b-3d9f-4389-8ea3-d1c554193914 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:26.533071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de1428-c08e-4baf-8b31-0ca0aea82efb None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:26.559077 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 489/982] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Jul 20 14:35:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:26.560309 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e599b-3d9f-4389-8ea3-d1c554193914 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:26.568419 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c037a4-0439-4355-85bd-84a71c8f84f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:26.569087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c037a4-0439-4355-85bd-84a71c8f84f5 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:26.569373 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c037a4-0439-4355-85bd-84a71c8f84f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:26.569670 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c037a4-0439-4355-85bd-84a71c8f84f5 None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:26.570347 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c037a4-0439-4355-85bd-84a71c8f84f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:26.571770 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c037a4-0439-4355-85bd-84a71c8f84f5 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:26.579078 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 494/983] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Jul 20 14:35:26 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 14:35:26.674970 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e599b-3d9f-4389-8ea3-d1c554193914 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:26.693534 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:26.715248 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec460aa-cbbd-49a0-be2b-512bf048e158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:26.715924 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec460aa-cbbd-49a0-be2b-512bf048e158 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:26.716192 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec460aa-cbbd-49a0-be2b-512bf048e158 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:26.716481 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec460aa-cbbd-49a0-be2b-512bf048e158 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:26.791367 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec460aa-cbbd-49a0-be2b-512bf048e158 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:26.792884 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec460aa-cbbd-49a0-be2b-512bf048e158 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:26.896163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 490/984] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:35:26 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 14:35:26.901929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c037a4-0439-4355-85bd-84a71c8f84f5 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:27.008302 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c037a4-0439-4355-85bd-84a71c8f84f5 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:27.042758 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2b829-d4e2-4de5-8eab-ad8d45236cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:27.043606 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2b829-d4e2-4de5-8eab-ad8d45236cde None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:27.043900 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2b829-d4e2-4de5-8eab-ad8d45236cde None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:27.044184 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2b829-d4e2-4de5-8eab-ad8d45236cde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:27.057235 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df2b829-d4e2-4de5-8eab-ad8d45236cde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:27.058713 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df2b829-d4e2-4de5-8eab-ad8d45236cde None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:27.065855 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:27.077929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-9df2b829-d4e2-4de5-8eab-ad8d45236cde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:27.077929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:27.077929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:27.077929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:27.077929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:27.077929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:27.077929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:27.077929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:27.077929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:27.077929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:27.077929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:27.077929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:27.077929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:35:27.077929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:35:27.077929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:35:27.078862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:35:27.078862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:27.078862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:27.078862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:27.078862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:27.078862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:27.078862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:27.078862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:27.078862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:27.078862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:27.078862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:27.078862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:27.078862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:27.078862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:27.078862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:27.078862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:27.078862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:27.079785 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:35:27.079785 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:35:27.079785 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:27.079785 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:27.083416 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 495/985] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:27.085196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec460aa-cbbd-49a0-be2b-512bf048e158 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:27.105175 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:27.183722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4487b-e815-42b1-8ec4-57b6939d5dec None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:27.184356 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4487b-e815-42b1-8ec4-57b6939d5dec None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:27.184740 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4487b-e815-42b1-8ec4-57b6939d5dec None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:27.186046 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 496/986] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:27.191433 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe4065-e0c0-411d-98f2-22fed334e513 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:27.191729 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe4065-e0c0-411d-98f2-22fed334e513 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:27.192014 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe4065-e0c0-411d-98f2-22fed334e513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:27.193193 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec460aa-cbbd-49a0-be2b-512bf048e158 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:27.211200 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-8ffe4065-e0c0-411d-98f2-22fed334e513 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:27.212463 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-8ffe4065-e0c0-411d-98f2-22fed334e513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:27.233003 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55895028-e4ee-48d6-b5dd-f60b840b4911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:27.233688 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55895028-e4ee-48d6-b5dd-f60b840b4911 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:27.233978 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55895028-e4ee-48d6-b5dd-f60b840b4911 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:27.234260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55895028-e4ee-48d6-b5dd-f60b840b4911 None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:27.234998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55895028-e4ee-48d6-b5dd-f60b840b4911 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:27.235339 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55895028-e4ee-48d6-b5dd-f60b840b4911 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:27.236754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55895028-e4ee-48d6-b5dd-f60b840b4911 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:27.248733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 491/987] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:35:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:27.254354 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55895028-e4ee-48d6-b5dd-f60b840b4911 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:27.287308 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 497/988] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:27 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 14:35:27.308559 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe4065-e0c0-411d-98f2-22fed334e513 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:27.339068 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:27.365986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55895028-e4ee-48d6-b5dd-f60b840b4911 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:27.415204 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01993fe-0ac4-41ed-a1f8-f3041926b183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:27.415863 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe4065-e0c0-411d-98f2-22fed334e513 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:27.416938 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01993fe-0ac4-41ed-a1f8-f3041926b183 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:27.417622 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01993fe-0ac4-41ed-a1f8-f3041926b183 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:27.418912 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01993fe-0ac4-41ed-a1f8-f3041926b183 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:27.428538 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01993fe-0ac4-41ed-a1f8-f3041926b183 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:27.430556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01993fe-0ac4-41ed-a1f8-f3041926b183 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:27.444896 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:27.448014 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-f01993fe-0ac4-41ed-a1f8-f3041926b183 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:35:27.448014 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:27.448014 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:27.448014 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:27.448014 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:27.448014 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:27.448014 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:27.448014 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:27.448014 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:27.448014 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:27.448014 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:27.448014 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:27.448014 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:35:27.448014 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:35:27.448014 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:35:27.449454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:35:27.449454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:27.449454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:27.449454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:27.449454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:27.449454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:27.449454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:27.449454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:27.449454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:27.449454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:27.449454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:27.449454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:27.449454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:27.449454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:27.449454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:27.449454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:27.449454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:27.450362 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:35:27.450362 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:35:27.450362 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:35:27.450362 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:35:27.450362 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:35:27.450362 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:35:27.450362 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:35:27.450362 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:27.450362 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 492/989] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:35:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:35:27.455222 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01993fe-0ac4-41ed-a1f8-f3041926b183 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:27.457046 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc5d25-b99e-4006-b395-2df27127ed15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:27.457953 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc5d25-b99e-4006-b395-2df27127ed15 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:27.458127 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc5d25-b99e-4006-b395-2df27127ed15 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:27.458401 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc5d25-b99e-4006-b395-2df27127ed15 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:27.459135 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffc5d25-b99e-4006-b395-2df27127ed15 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:27.460524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffc5d25-b99e-4006-b395-2df27127ed15 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:27.472708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-4ffc5d25-b99e-4006-b395-2df27127ed15 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 14:35:27.472708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:27.472708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 14:35:27.472708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 14:35:27.472708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 14:35:27.472708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 14:35:27.472708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 14:35:27.472708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:27.472708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 14:35:27.472708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:27.472708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 14:35:27.472708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:27.472708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:27.472708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:27.472708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:27.472708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:27.472708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:27.473764 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:27.473764 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:27.473764 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:27.473764 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:27.473764 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:27.473764 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:27.473764 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 14:35:27.473764 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 14:35:27.473764 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 14:35:27.473764 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 14:35:27.473764 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:27.473764 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:27.473764 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:27.473764 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:27.473764 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:27.473764 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:27.474760 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:27.474760 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:27.474760 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:27.474760 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:27.474760 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:27.474760 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:27.474760 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:27.474760 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:27.474760 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:27.474760 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:27.474760 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 14:35:27.474760 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:27.474760 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 14:35:27.474760 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:27.474760 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 498/990] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:35:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:35:27.480957 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc5d25-b99e-4006-b395-2df27127ed15 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:27.570939 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01993fe-0ac4-41ed-a1f8-f3041926b183 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:27.594706 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc5d25-b99e-4006-b395-2df27127ed15 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:27.610077 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae029c-974e-4f2e-8d11-2d06c069c0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:27.610754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae029c-974e-4f2e-8d11-2d06c069c0d7 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:27.611066 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae029c-974e-4f2e-8d11-2d06c069c0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:27.611456 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae029c-974e-4f2e-8d11-2d06c069c0d7 None admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:27.612267 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae029c-974e-4f2e-8d11-2d06c069c0d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:27.612581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae029c-974e-4f2e-8d11-2d06c069c0d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:27.614018 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae029c-974e-4f2e-8d11-2d06c069c0d7 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:27.616612 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:27.631793 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 493/991] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:35:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:27.633251 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae029c-974e-4f2e-8d11-2d06c069c0d7 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:27.634115 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8593ba-5936-4d59-a28e-09c2e4bb3644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:27.634973 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8593ba-5936-4d59-a28e-09c2e4bb3644 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:27.635413 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8593ba-5936-4d59-a28e-09c2e4bb3644 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:27.635828 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8593ba-5936-4d59-a28e-09c2e4bb3644 None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:27.636673 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8593ba-5936-4d59-a28e-09c2e4bb3644 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:27.637147 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8593ba-5936-4d59-a28e-09c2e4bb3644 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:27.638651 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8593ba-5936-4d59-a28e-09c2e4bb3644 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:27.648540 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 499/992] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Jul 20 14:35:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:27.649608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8593ba-5936-4d59-a28e-09c2e4bb3644 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:27.707278 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:27.747527 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae029c-974e-4f2e-8d11-2d06c069c0d7 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:27.757688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8593ba-5936-4d59-a28e-09c2e4bb3644 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:27.786946 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54445126-cc9f-4266-9eb9-202a627435dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:27.787654 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54445126-cc9f-4266-9eb9-202a627435dc None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:27.787943 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54445126-cc9f-4266-9eb9-202a627435dc None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:27.788223 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54445126-cc9f-4266-9eb9-202a627435dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:27.789085 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54445126-cc9f-4266-9eb9-202a627435dc None admin] RBAC: Adding query filter params (role.id=59c1c6b5011a4faea5f02d1961352d67, scope.project.id=59333e4f593346e184877d6bd9d22026, user.id=2f8cf6d908314281a3ddc9d9513d39c2) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:27.789404 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54445126-cc9f-4266-9eb9-202a627435dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:27.790943 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54445126-cc9f-4266-9eb9-202a627435dc None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:27.793982 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cac9c6-526e-4b96-be30-9f50f94664eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:27.794672 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cac9c6-526e-4b96-be30-9f50f94664eb None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:27.794954 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cac9c6-526e-4b96-be30-9f50f94664eb None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:27.795270 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cac9c6-526e-4b96-be30-9f50f94664eb None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:27.795999 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cac9c6-526e-4b96-be30-9f50f94664eb None admin] RBAC: Adding query filter params (type=compute) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:27.796311 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cac9c6-526e-4b96-be30-9f50f94664eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:27.798048 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cac9c6-526e-4b96-be30-9f50f94664eb None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:27.798641 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 494/993] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:35:27 2022] GET /identity/v3/role_assignments?user.id=2f8cf6d908314281a3ddc9d9513d39c2&scope.project.id=59333e4f593346e184877d6bd9d22026&role.id=59c1c6b5011a4faea5f02d1961352d67 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:27.804918 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 500/994] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Jul 20 14:35:27 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:27.810775 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54445126-cc9f-4266-9eb9-202a627435dc None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:27.880200 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:27.927755 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54445126-cc9f-4266-9eb9-202a627435dc None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:27.967140 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d84521a-0839-49d3-abdb-b6a18ac366ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:27.967897 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84521a-0839-49d3-abdb-b6a18ac366ae None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:27.968187 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84521a-0839-49d3-abdb-b6a18ac366ae None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:27.968467 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84521a-0839-49d3-abdb-b6a18ac366ae None admin] PATH_INFO: `/v3/endpoints` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:27.969265 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d84521a-0839-49d3-abdb-b6a18ac366ae None admin] RBAC: Adding query filter params (interface=public, service_id=23f8e884fea84464b7bfad384042fa4c, region_id=RegionOne) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:27.969581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d84521a-0839-49d3-abdb-b6a18ac366ae None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:27.970979 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d84521a-0839-49d3-abdb-b6a18ac366ae None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:27.983219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 495/995] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Jul 20 14:35:27 2022] GET /identity/v3/endpoints?service_id=23f8e884fea84464b7bfad384042fa4c&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:27.984904 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cac9c6-526e-4b96-be30-9f50f94664eb None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:28.095297 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cac9c6-526e-4b96-be30-9f50f94664eb None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:28.115887 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:28.132086 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec3e88-4bf8-4749-83d9-d44f9f5cb047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:28.132694 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec3e88-4bf8-4749-83d9-d44f9f5cb047 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:28.132974 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec3e88-4bf8-4749-83d9-d44f9f5cb047 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:28.133244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec3e88-4bf8-4749-83d9-d44f9f5cb047 None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:28.133932 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ec3e88-4bf8-4749-83d9-d44f9f5cb047 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:28.135325 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ec3e88-4bf8-4749-83d9-d44f9f5cb047 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:28.142209 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 501/996] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Jul 20 14:35:27 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:35:28.794451 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52721d8-0766-4cba-b76b-855eac1e157c None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:28.795003 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52721d8-0766-4cba-b76b-855eac1e157c None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:28.795003 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52721d8-0766-4cba-b76b-855eac1e157c None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:28.796644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 496/997] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:28.802392 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe01d6-0395-4b44-8a7b-84f70b90d55f None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:28.803290 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe01d6-0395-4b44-8a7b-84f70b90d55f None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:28.803443 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe01d6-0395-4b44-8a7b-84f70b90d55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:28.821022 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-aefe01d6-0395-4b44-8a7b-84f70b90d55f None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:28.822406 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-aefe01d6-0395-4b44-8a7b-84f70b90d55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:28.895496 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 502/998] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:28 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:35:28.918691 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52721d8-0766-4cba-b76b-855eac1e157c None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:29.027556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52721d8-0766-4cba-b76b-855eac1e157c None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:29.050912 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:29.062676 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6603ba-03a2-4d04-8c96-37c9ae270610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:29.063521 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6603ba-03a2-4d04-8c96-37c9ae270610 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:29.063803 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6603ba-03a2-4d04-8c96-37c9ae270610 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:29.064082 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6603ba-03a2-4d04-8c96-37c9ae270610 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:29.064741 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6603ba-03a2-4d04-8c96-37c9ae270610 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:29.066121 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6603ba-03a2-4d04-8c96-37c9ae270610 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:29.078195 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-bb6603ba-03a2-4d04-8c96-37c9ae270610 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 14:35:29.078195 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:29.078195 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 14:35:29.078195 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 14:35:29.078195 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 14:35:29.078195 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 14:35:29.078195 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 14:35:29.078195 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:29.078195 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 14:35:29.078195 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:29.078195 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 14:35:29.078195 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:29.078195 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:29.078195 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:29.078195 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:29.078195 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:29.078195 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:29.079336 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:29.079336 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:29.079336 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:29.079336 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:29.079336 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:29.079336 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:29.079336 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 14:35:29.079336 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 14:35:29.079336 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 14:35:29.079336 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 14:35:29.079336 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:29.079336 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:29.079336 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:29.079336 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:29.079336 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:29.079336 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:29.080257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:29.080257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:29.080257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:29.080257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:29.080257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:29.080257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:29.080257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:29.080257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:29.080257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:29.080257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:29.080257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 14:35:29.080257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:29.080257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 14:35:29.080257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:29.080257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 497/999] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Jul 20 14:35:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:29.085411 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe01d6-0395-4b44-8a7b-84f70b90d55f None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:29.189942 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe01d6-0395-4b44-8a7b-84f70b90d55f None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:29.205321 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:29.225759 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac14a4-b173-4f0b-bc36-3b4f05309be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:29.226622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac14a4-b173-4f0b-bc36-3b4f05309be5 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:29.226897 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac14a4-b173-4f0b-bc36-3b4f05309be5 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:29.227205 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac14a4-b173-4f0b-bc36-3b4f05309be5 None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:29.228028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ac14a4-b173-4f0b-bc36-3b4f05309be5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:29.228338 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ac14a4-b173-4f0b-bc36-3b4f05309be5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:29.229725 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ac14a4-b173-4f0b-bc36-3b4f05309be5 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:29.235135 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 503/1000] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Jul 20 14:35:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:29.241281 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6603ba-03a2-4d04-8c96-37c9ae270610 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:29.350608 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6603ba-03a2-4d04-8c96-37c9ae270610 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:29.385881 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20222739-a152-4575-979b-62c34d3513a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:29.386599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20222739-a152-4575-979b-62c34d3513a8 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:29.386888 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20222739-a152-4575-979b-62c34d3513a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:29.387200 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20222739-a152-4575-979b-62c34d3513a8 None admin] PATH_INFO: `/v3/services` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:29.387919 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20222739-a152-4575-979b-62c34d3513a8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:29.388228 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20222739-a152-4575-979b-62c34d3513a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:29.389615 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20222739-a152-4575-979b-62c34d3513a8 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:29.394742 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 498/1001] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Jul 20 14:35:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:29.400797 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac14a4-b173-4f0b-bc36-3b4f05309be5 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:29.516201 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac14a4-b173-4f0b-bc36-3b4f05309be5 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:29.553850 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab2550-812a-4fcd-b95b-a9cc5f43e53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:29.555009 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab2550-812a-4fcd-b95b-a9cc5f43e53d None admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:29.555529 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab2550-812a-4fcd-b95b-a9cc5f43e53d None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:29.555993 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab2550-812a-4fcd-b95b-a9cc5f43e53d None admin] PATH_INFO: `/v3/endpoints` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:29.556971 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab2550-812a-4fcd-b95b-a9cc5f43e53d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:29.558571 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab2550-812a-4fcd-b95b-a9cc5f43e53d None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:29.575654 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 504/1002] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Jul 20 14:35:29 2022] POST /identity/v3/endpoints => generated 325 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:35:29.907936 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d278510-b6d0-4826-b600-bc0b8d7c65bb None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:29.908503 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d278510-b6d0-4826-b600-bc0b8d7c65bb None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:29.908503 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d278510-b6d0-4826-b600-bc0b8d7c65bb None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:29.909847 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 499/1003] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:29.915522 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e806c0-ca39-4757-966f-a756015192aa None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:29.915966 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e806c0-ca39-4757-966f-a756015192aa None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:29.916050 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e806c0-ca39-4757-966f-a756015192aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:29.934693 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-e7e806c0-ca39-4757-966f-a756015192aa None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:29.936037 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e7e806c0-ca39-4757-966f-a756015192aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:29.971654 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:30.021732 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 505/1004] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:29 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:35:30.044571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d278510-b6d0-4826-b600-bc0b8d7c65bb None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:30.103829 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:30.159355 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d278510-b6d0-4826-b600-bc0b8d7c65bb None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:30.205561 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57eaab-7cb1-4262-9b1b-578a55d2cb50 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:30.205967 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57eaab-7cb1-4262-9b1b-578a55d2cb50 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:30.206168 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57eaab-7cb1-4262-9b1b-578a55d2cb50 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:30.206600 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6318e50b-8750-4313-a403-664cfc26755f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:30.207443 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318e50b-8750-4313-a403-664cfc26755f None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:30.207742 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318e50b-8750-4313-a403-664cfc26755f None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:30.207860 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 506/1005] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 14:35:30.208041 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318e50b-8750-4313-a403-664cfc26755f None admin] PATH_INFO: `/v3/services/image` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:30.208711 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6318e50b-8750-4313-a403-664cfc26755f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:30.210610 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6318e50b-8750-4313-a403-664cfc26755f None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:30.212959 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db723292-1306-4262-b9b8-f0e7edc4b24e None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:30.213250 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db723292-1306-4262-b9b8-f0e7edc4b24e None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:30.213525 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db723292-1306-4262-b9b8-f0e7edc4b24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:30.229853 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-6318e50b-8750-4313-a403-664cfc26755f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 14:35:30.229853 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:30.229853 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 14:35:30.229853 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 14:35:30.229853 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 14:35:30.229853 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 14:35:30.229853 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 14:35:30.229853 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:30.229853 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 14:35:30.229853 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:30.229853 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 14:35:30.229853 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:30.229853 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:30.229853 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:30.229853 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:30.229853 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:30.229853 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:30.230942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:30.230942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:30.230942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:30.230942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:30.230942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:30.230942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:30.230942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 14:35:30.230942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 14:35:30.230942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 14:35:30.230942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 14:35:30.230942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:30.230942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:30.230942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:30.230942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:30.230942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:30.230942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:30.231846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:30.231846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:30.231846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:30.231846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:30.231846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:30.231846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:30.231846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:30.231846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:30.231846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:30.231846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:30.231846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 14:35:30.231846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:30.231846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 14:35:30.231846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:30.233236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-db723292-1306-4262-b9b8-f0e7edc4b24e None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:30.233236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-db723292-1306-4262-b9b8-f0e7edc4b24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:30.235389 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 500/1006] 10.222.0.15 () {56 vars in 1105 bytes} [Wed Jul 20 14:35:30 2022] GET /identity/v3/services/image => generated 86 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:30.237098 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6318e50b-8750-4313-a403-664cfc26755f None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:30.303455 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 507/1007] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:30 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 14:35:30.326790 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db723292-1306-4262-b9b8-f0e7edc4b24e None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:30.342535 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6318e50b-8750-4313-a403-664cfc26755f None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:30.360048 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:30.381587 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82637c2c-7f5a-4b24-b2c2-7f0bd784f214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:30.382267 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82637c2c-7f5a-4b24-b2c2-7f0bd784f214 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:30.382551 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82637c2c-7f5a-4b24-b2c2-7f0bd784f214 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:30.382839 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82637c2c-7f5a-4b24-b2c2-7f0bd784f214 None admin] PATH_INFO: `/v3/services` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:30.383733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82637c2c-7f5a-4b24-b2c2-7f0bd784f214 None admin] RBAC: Adding query filter params (name=image) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:30.384201 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82637c2c-7f5a-4b24-b2c2-7f0bd784f214 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:30.385965 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82637c2c-7f5a-4b24-b2c2-7f0bd784f214 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:30.391580 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 501/1008] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:30.397487 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82637c2c-7f5a-4b24-b2c2-7f0bd784f214 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:30.428196 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:30.437166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db723292-1306-4262-b9b8-f0e7edc4b24e None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:30.480417 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005e342-4ddf-4f95-8d2b-885686109da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:30.481276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005e342-4ddf-4f95-8d2b-885686109da2 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:30.481574 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005e342-4ddf-4f95-8d2b-885686109da2 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:30.481874 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005e342-4ddf-4f95-8d2b-885686109da2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:30.482597 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005e342-4ddf-4f95-8d2b-885686109da2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:30.484027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005e342-4ddf-4f95-8d2b-885686109da2 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:30.500655 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-1005e342-4ddf-4f95-8d2b-885686109da2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 14:35:30.500655 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:30.500655 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 14:35:30.500655 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 14:35:30.500655 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 14:35:30.500655 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 14:35:30.500655 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 14:35:30.500655 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:30.500655 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 14:35:30.500655 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:30.500655 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 14:35:30.500655 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:30.500655 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:30.500655 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:30.500655 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:30.500655 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:30.500655 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:30.501832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:30.501832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:30.501832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:30.501832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:30.501832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:30.501832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:30.501832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 14:35:30.501832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 14:35:30.501832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 14:35:30.501832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 14:35:30.501832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:30.501832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:30.501832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:30.501832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:30.501832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:30.501832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:30.502701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:30.502701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:30.502701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:30.502701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:30.502701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:30.502701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:30.502701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:30.502701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:30.502701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:30.502701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:30.502701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 14:35:30.502701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:30.502701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 14:35:30.502701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:30.502701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 508/1009] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:35:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:35:30.511213 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005e342-4ddf-4f95-8d2b-885686109da2 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:30.512920 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82637c2c-7f5a-4b24-b2c2-7f0bd784f214 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:30.555384 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd79f1c-965e-4caf-8921-4f851179be63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:30.556452 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd79f1c-965e-4caf-8921-4f851179be63 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:30.556969 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd79f1c-965e-4caf-8921-4f851179be63 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:30.557420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd79f1c-965e-4caf-8921-4f851179be63 None admin] PATH_INFO: `/v3/services` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:30.558388 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd79f1c-965e-4caf-8921-4f851179be63 None admin] RBAC: Adding query filter params (type=image) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:30.558867 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd79f1c-965e-4caf-8921-4f851179be63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:30.561379 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd79f1c-965e-4caf-8921-4f851179be63 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:30.567306 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 502/1010] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:30 2022] GET /identity/v3/services?type=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:30.644784 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005e342-4ddf-4f95-8d2b-885686109da2 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:30.682882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba5a99-37da-4595-bbae-886b721b970f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:30.683560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba5a99-37da-4595-bbae-886b721b970f None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:30.683864 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba5a99-37da-4595-bbae-886b721b970f None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:30.684140 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba5a99-37da-4595-bbae-886b721b970f None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:30.684907 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ba5a99-37da-4595-bbae-886b721b970f None admin] RBAC: Adding query filter params (name=compute) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:30.685238 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ba5a99-37da-4595-bbae-886b721b970f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:30.686671 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ba5a99-37da-4595-bbae-886b721b970f None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:30.692564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 509/1011] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Jul 20 14:35:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:30.698562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd79f1c-965e-4caf-8921-4f851179be63 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:30.754992 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:30.813719 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd79f1c-965e-4caf-8921-4f851179be63 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:30.850420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f6162a-f416-4299-bd3c-2f936934dbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:30.851150 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6162a-f416-4299-bd3c-2f936934dbde None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:30.851436 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6162a-f416-4299-bd3c-2f936934dbde None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:30.851745 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6162a-f416-4299-bd3c-2f936934dbde None admin] PATH_INFO: `/v3/services` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:30.852482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f6162a-f416-4299-bd3c-2f936934dbde None admin] RBAC: Adding query filter params (type=compute) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:30.852796 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f6162a-f416-4299-bd3c-2f936934dbde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:30.854257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f6162a-f416-4299-bd3c-2f936934dbde None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:30.864219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 503/1012] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Jul 20 14:35:30 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:30.865621 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba5a99-37da-4595-bbae-886b721b970f None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:30.892246 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:30.974561 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba5a99-37da-4595-bbae-886b721b970f None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:31.012954 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a6f2f7-e78b-4a60-851a-f930d12d4507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:31.014046 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6f2f7-e78b-4a60-851a-f930d12d4507 None admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:31.014577 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6f2f7-e78b-4a60-851a-f930d12d4507 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:31.015120 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6f2f7-e78b-4a60-851a-f930d12d4507 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:31.016194 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a6f2f7-e78b-4a60-851a-f930d12d4507 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:31.017905 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a6f2f7-e78b-4a60-851a-f930d12d4507 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:31.039421 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 510/1013] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Jul 20 14:35:30 2022] POST /identity/v3/endpoints => generated 334 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:35:31.642553 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69571737-a88a-4817-8bf5-ea06547ae07c None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:31.643320 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69571737-a88a-4817-8bf5-ea06547ae07c None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:31.643320 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69571737-a88a-4817-8bf5-ea06547ae07c None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:31.644682 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 504/1014] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:31.649899 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f049778-6a01-4183-b869-11c808207d9a None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:31.650192 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f049778-6a01-4183-b869-11c808207d9a None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:31.650469 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f049778-6a01-4183-b869-11c808207d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:31.667703 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-6f049778-6a01-4183-b869-11c808207d9a None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:31.668899 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-6f049778-6a01-4183-b869-11c808207d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:31.746699 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 511/1015] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:31 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:35:31.769354 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69571737-a88a-4817-8bf5-ea06547ae07c None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:31.876100 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69571737-a88a-4817-8bf5-ea06547ae07c None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:31.885809 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:31.914685 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4dacdc-6d64-4a73-b953-0ae6a1980c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:31.915765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4dacdc-6d64-4a73-b953-0ae6a1980c5a None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:31.916190 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4dacdc-6d64-4a73-b953-0ae6a1980c5a None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:31.916471 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4dacdc-6d64-4a73-b953-0ae6a1980c5a None admin] PATH_INFO: `/v3/domains/default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:31.917669 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4dacdc-6d64-4a73-b953-0ae6a1980c5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:31.919147 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4dacdc-6d64-4a73-b953-0ae6a1980c5a None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:31.925829 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 505/1016] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:31.927284 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f049778-6a01-4183-b869-11c808207d9a None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:31.930673 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:32.028318 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f049778-6a01-4183-b869-11c808207d9a None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:32.062687 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e356ff-d177-4c65-8f3d-cdcb96293b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:32.063397 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e356ff-d177-4c65-8f3d-cdcb96293b62 None admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:32.063688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e356ff-d177-4c65-8f3d-cdcb96293b62 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:32.063968 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e356ff-d177-4c65-8f3d-cdcb96293b62 None admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:32.064700 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e356ff-d177-4c65-8f3d-cdcb96293b62 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:32.066135 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e356ff-d177-4c65-8f3d-cdcb96293b62 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:32.092931 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 512/1017] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Jul 20 14:35:31 2022] POST /identity/v3/projects => generated 311 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:35:32.626660 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac98f8-b67c-4d13-9ecb-38fdd3909e7f None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:32.627304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac98f8-b67c-4d13-9ecb-38fdd3909e7f None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:32.627743 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac98f8-b67c-4d13-9ecb-38fdd3909e7f None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:32.629188 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 506/1018] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:32.634894 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccab1a-b33b-4997-bc57-1ff22c7a177e None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:32.635203 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccab1a-b33b-4997-bc57-1ff22c7a177e None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:32.635499 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccab1a-b33b-4997-bc57-1ff22c7a177e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:32.653331 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-18ccab1a-b33b-4997-bc57-1ff22c7a177e None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:32.654546 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-18ccab1a-b33b-4997-bc57-1ff22c7a177e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:32.705808 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:32.724482 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 513/1019] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:32 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:35:32.747106 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac98f8-b67c-4d13-9ecb-38fdd3909e7f None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:32.848748 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac98f8-b67c-4d13-9ecb-38fdd3909e7f None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:32.884398 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf8804-7455-4d4f-b095-c45f407449b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:32.885073 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf8804-7455-4d4f-b095-c45f407449b1 None admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:32.885355 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf8804-7455-4d4f-b095-c45f407449b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:32.885625 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf8804-7455-4d4f-b095-c45f407449b1 None admin] PATH_INFO: `/v3/services` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:32.886375 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaf8804-7455-4d4f-b095-c45f407449b1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:32.887781 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaf8804-7455-4d4f-b095-c45f407449b1 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:32.903949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 507/1020] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Jul 20 14:35:32 2022] POST /identity/v3/services => generated 242 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:35:33.214530 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd4acd-3d79-44bf-aae3-b515b2e55b27 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:33.215126 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd4acd-3d79-44bf-aae3-b515b2e55b27 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:33.215126 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd4acd-3d79-44bf-aae3-b515b2e55b27 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:33.216644 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 514/1021] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:33.221946 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee16a56-e1fc-4071-8058-dd0124691ffb None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:33.222236 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee16a56-e1fc-4071-8058-dd0124691ffb None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:33.222518 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee16a56-e1fc-4071-8058-dd0124691ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:33.241171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-2ee16a56-e1fc-4071-8058-dd0124691ffb None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:33.242375 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-2ee16a56-e1fc-4071-8058-dd0124691ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:33.276171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:33.324934 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 508/1022] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:35:33.350021 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd4acd-3d79-44bf-aae3-b515b2e55b27 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:33.465712 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd4acd-3d79-44bf-aae3-b515b2e55b27 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:33.505292 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad978d1-1f1d-4f55-a43c-4ef73af3d4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:33.506165 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad978d1-1f1d-4f55-a43c-4ef73af3d4e5 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:33.506452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad978d1-1f1d-4f55-a43c-4ef73af3d4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:33.506730 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad978d1-1f1d-4f55-a43c-4ef73af3d4e5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:33.517135 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad978d1-1f1d-4f55-a43c-4ef73af3d4e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:35:33.525503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-8ad978d1-1f1d-4f55-a43c-4ef73af3d4e5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 14:35:33.525503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:33.525503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:33.525503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:33.525503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:33.525503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:33.525503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:33.525503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:33.525503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:33.525503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:33.525503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:33.525503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:33.525503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:35:33.525503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:35:33.525503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:35:33.525503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:35:33.526513 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:35:33.526513 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:35:33.526513 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:35:33.526513 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:35:33.526513 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:33.526513 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:33.526513 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:33.526513 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:33.526513 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:33.526513 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:33.526513 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:33.526513 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:33.526513 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:33.526513 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:33.526513 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:33.526513 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:33.527487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:33.527487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:33.527487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:33.527487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:33.527487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:35:33.527487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:35:33.527487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:35:33.527487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:35:33.527487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:35:33.527487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:35:33.527487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 14:35:33.527487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:33.531419 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 515/1023] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Jul 20 14:35:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:33.533073 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee16a56-e1fc-4071-8058-dd0124691ffb None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:33.636096 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee16a56-e1fc-4071-8058-dd0124691ffb None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:33.671520 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def71e40-1de2-4ca0-9769-62b3120f43b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:33.672508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def71e40-1de2-4ca0-9769-62b3120f43b6 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:33.672792 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def71e40-1de2-4ca0-9769-62b3120f43b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:33.673070 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def71e40-1de2-4ca0-9769-62b3120f43b6 None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:33.673826 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def71e40-1de2-4ca0-9769-62b3120f43b6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:33.674137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def71e40-1de2-4ca0-9769-62b3120f43b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:33.675570 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def71e40-1de2-4ca0-9769-62b3120f43b6 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:33.683855 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 509/1024] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Jul 20 14:35:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:33.689670 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad978d1-1f1d-4f55-a43c-4ef73af3d4e5 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:33.737538 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:33.793802 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad978d1-1f1d-4f55-a43c-4ef73af3d4e5 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:33.829086 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc450b-811c-4003-af52-8e27994da5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:33.829889 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc450b-811c-4003-af52-8e27994da5cb None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:33.830212 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc450b-811c-4003-af52-8e27994da5cb None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:33.830454 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc450b-811c-4003-af52-8e27994da5cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:33.842200 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc450b-811c-4003-af52-8e27994da5cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:33.843699 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc450b-811c-4003-af52-8e27994da5cb None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:33.861202 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-72cc450b-811c-4003-af52-8e27994da5cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:33.861202 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:33.861202 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:33.861202 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:33.861202 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:33.861202 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:33.861202 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:33.861202 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:33.861202 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:33.861202 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:33.861202 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:33.861202 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:33.861202 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:35:33.861202 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:35:33.861202 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:35:33.862174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:35:33.862174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:33.862174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:33.862174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:33.862174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:33.862174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:33.862174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:33.862174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:33.862174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:33.862174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:33.862174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:33.862174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:33.862174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:33.862174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:33.862174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:33.862174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:33.862174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:33.869610 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:35:33.869610 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:35:33.869610 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:33.869610 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:33.869610 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 516/1025] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:35:33.869919 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def71e40-1de2-4ca0-9769-62b3120f43b6 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:33.897435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:33.977208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def71e40-1de2-4ca0-9769-62b3120f43b6 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:34.011960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6c87f-b8d8-4131-b8fe-006fb9db5934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:34.012555 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6c87f-b8d8-4131-b8fe-006fb9db5934 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:34.012841 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6c87f-b8d8-4131-b8fe-006fb9db5934 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:34.013114 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6c87f-b8d8-4131-b8fe-006fb9db5934 None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:34.013833 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e6c87f-b8d8-4131-b8fe-006fb9db5934 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:34.014135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e6c87f-b8d8-4131-b8fe-006fb9db5934 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:34.015642 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e6c87f-b8d8-4131-b8fe-006fb9db5934 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:34.034108 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 510/1026] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:34.035421 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc450b-811c-4003-af52-8e27994da5cb None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:34.135316 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:34.140162 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc450b-811c-4003-af52-8e27994da5cb None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:34.175714 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b73ea-2d65-4b8e-9f11-4f1d384d6faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:34.176522 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b73ea-2d65-4b8e-9f11-4f1d384d6faf None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:34.176812 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b73ea-2d65-4b8e-9f11-4f1d384d6faf None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:34.177085 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b73ea-2d65-4b8e-9f11-4f1d384d6faf None admin] PATH_INFO: `/v3/users/nova` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:34.183310 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07a25a-6a06-4fcd-a4dd-dd3e3c9dc685 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:34.183596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07a25a-6a06-4fcd-a4dd-dd3e3c9dc685 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:34.183872 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07a25a-6a06-4fcd-a4dd-dd3e3c9dc685 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:34.185390 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 511/1027] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:34.185495 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b73ea-2d65-4b8e-9f11-4f1d384d6faf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:34.186900 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b73ea-2d65-4b8e-9f11-4f1d384d6faf None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:34.190151 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c49158-af46-40f8-befb-ec67dde466f3 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:34.190433 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c49158-af46-40f8-befb-ec67dde466f3 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:34.190703 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c49158-af46-40f8-befb-ec67dde466f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:34.201511 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-da8b73ea-2d65-4b8e-9f11-4f1d384d6faf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 14:35:34.201511 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:34.201511 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:34.201511 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:34.201511 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:34.201511 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:34.201511 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:34.201511 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:34.201511 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:34.201511 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:34.201511 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:34.201511 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:34.201511 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:35:34.201511 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:35:34.201511 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:35:34.201511 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:35:34.203276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:34.203276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:34.203276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:35:34.203276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:35:34.203276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:35:34.203276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:35:34.203276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:34.203276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:34.203276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:34.203276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:34.203276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:34.203276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:34.203276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:34.203276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:34.203276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:34.203276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:34.203276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:34.203276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:34.204235 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:34.204235 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:34.204235 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:35:34.204235 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:35:34.204235 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:35:34.204235 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:35:34.204235 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:35:34.204235 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:35:34.204235 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 14:35:34.204235 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:34.204235 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 517/1028] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Jul 20 14:35:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:34.208865 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b73ea-2d65-4b8e-9f11-4f1d384d6faf None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:34.211723 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-11c49158-af46-40f8-befb-ec67dde466f3 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:34.212923 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-11c49158-af46-40f8-befb-ec67dde466f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:34.285849 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 512/1029] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:34 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 14:35:34.310968 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c49158-af46-40f8-befb-ec67dde466f3 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:34.322618 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b73ea-2d65-4b8e-9f11-4f1d384d6faf None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:34.329672 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:34.364049 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88039698-ab29-4833-a38c-2d3140420297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:34.364751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88039698-ab29-4833-a38c-2d3140420297 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:34.365087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88039698-ab29-4833-a38c-2d3140420297 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:34.365370 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88039698-ab29-4833-a38c-2d3140420297 None admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:34.366258 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88039698-ab29-4833-a38c-2d3140420297 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:34.366570 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88039698-ab29-4833-a38c-2d3140420297 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:34.368289 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88039698-ab29-4833-a38c-2d3140420297 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:34.387395 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 518/1030] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Jul 20 14:35:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:34.392939 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88039698-ab29-4833-a38c-2d3140420297 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:34.412373 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:34.431699 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c49158-af46-40f8-befb-ec67dde466f3 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:34.468972 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563e34c-0a51-4aba-92ea-a9306de0194a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:34.469773 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563e34c-0a51-4aba-92ea-a9306de0194a None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:34.470214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563e34c-0a51-4aba-92ea-a9306de0194a None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:34.470499 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563e34c-0a51-4aba-92ea-a9306de0194a None admin] PATH_INFO: `/v3/domains/default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:34.472552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2563e34c-0a51-4aba-92ea-a9306de0194a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:34.474021 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2563e34c-0a51-4aba-92ea-a9306de0194a None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:34.478457 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 513/1031] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:34.483410 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563e34c-0a51-4aba-92ea-a9306de0194a None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:34.506739 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88039698-ab29-4833-a38c-2d3140420297 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:34.543391 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50d937-619f-40bb-bd07-5e2b8eca312e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:34.544060 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50d937-619f-40bb-bd07-5e2b8eca312e None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:34.544348 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50d937-619f-40bb-bd07-5e2b8eca312e None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:34.544630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50d937-619f-40bb-bd07-5e2b8eca312e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:34.582343 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:34.601962 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563e34c-0a51-4aba-92ea-a9306de0194a None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:34.626180 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e50d937-619f-40bb-bd07-5e2b8eca312e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:34.627777 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e50d937-619f-40bb-bd07-5e2b8eca312e None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:34.641125 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5d68db-1abb-444b-ab22-a0ee737c5e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:34.642500 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d68db-1abb-444b-ab22-a0ee737c5e80 None admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:34.642996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d68db-1abb-444b-ab22-a0ee737c5e80 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:34.643517 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d68db-1abb-444b-ab22-a0ee737c5e80 None admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:34.644851 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5d68db-1abb-444b-ab22-a0ee737c5e80 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:34.647194 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:34.647637 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5d68db-1abb-444b-ab22-a0ee737c5e80 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:34.670827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 514/1032] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jul 20 14:35:34 2022] POST /identity/v3/users => generated 285 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:35:34.751502 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 519/1033] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:35:34 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 14:35:34.753929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5d68db-1abb-444b-ab22-a0ee737c5e80 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:34.869065 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5d68db-1abb-444b-ab22-a0ee737c5e80 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:34.906365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5f9d7-4aee-4f49-969e-6581df83d7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:34.907243 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5f9d7-4aee-4f49-969e-6581df83d7a9 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:34.907543 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5f9d7-4aee-4f49-969e-6581df83d7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:34.907818 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5f9d7-4aee-4f49-969e-6581df83d7a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:34.915739 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:34.920250 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a5f9d7-4aee-4f49-969e-6581df83d7a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:34.922399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a5f9d7-4aee-4f49-969e-6581df83d7a9 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:34.941333 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-d8a5f9d7-4aee-4f49-969e-6581df83d7a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:34.941333 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:34.941333 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:34.941333 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:34.941333 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:34.941333 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:34.941333 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:34.941333 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:34.941333 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:34.941333 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:34.941333 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:34.941333 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:34.941333 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:35:34.941333 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:35:34.941333 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:35:34.942372 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:35:34.942372 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:34.942372 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:34.942372 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:34.942372 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:34.942372 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:34.942372 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:34.942372 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:34.942372 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:34.942372 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:34.942372 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:34.942372 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:34.942372 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:34.942372 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:34.942372 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:34.942372 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:34.942372 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:34.943940 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:35:34.943940 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:35:34.943940 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:34.943940 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:34.943940 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 515/1034] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:34.950520 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50d937-619f-40bb-bd07-5e2b8eca312e None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:34.952599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a809e-b3fb-438c-8e85-c124188b4acc None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:34.953125 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a809e-b3fb-438c-8e85-c124188b4acc None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:34.953254 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a809e-b3fb-438c-8e85-c124188b4acc None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:34.954735 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 516/1035] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:34.959853 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e215fa-a8e8-4f43-b0ff-94ad62a1f98f None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:34.960138 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e215fa-a8e8-4f43-b0ff-94ad62a1f98f None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:34.960407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e215fa-a8e8-4f43-b0ff-94ad62a1f98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:34.978284 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-44e215fa-a8e8-4f43-b0ff-94ad62a1f98f None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:34.979523 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-44e215fa-a8e8-4f43-b0ff-94ad62a1f98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:35.050041 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 517/1036] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:34 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 14:35:35.059217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50d937-619f-40bb-bd07-5e2b8eca312e None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:35.072553 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e215fa-a8e8-4f43-b0ff-94ad62a1f98f None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:35.077523 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:35.097743 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4ef03-8977-4e06-8527-7db0331fa047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:35.098429 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4ef03-8977-4e06-8527-7db0331fa047 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:35.098695 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4ef03-8977-4e06-8527-7db0331fa047 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:35.098971 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4ef03-8977-4e06-8527-7db0331fa047 None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:35.099901 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf4ef03-8977-4e06-8527-7db0331fa047 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:35.100279 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf4ef03-8977-4e06-8527-7db0331fa047 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:35.101666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf4ef03-8977-4e06-8527-7db0331fa047 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:35.118355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 520/1037] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:35.120240 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4ef03-8977-4e06-8527-7db0331fa047 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:35.178148 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e215fa-a8e8-4f43-b0ff-94ad62a1f98f None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:35.199655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:35.218841 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d956522-a44e-4398-a073-458d4a84571f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:35.220049 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d956522-a44e-4398-a073-458d4a84571f None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:35.220597 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d956522-a44e-4398-a073-458d4a84571f None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:35.221196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d956522-a44e-4398-a073-458d4a84571f None admin] PATH_INFO: `/v3/services/image` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:35.222088 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d956522-a44e-4398-a073-458d4a84571f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:35.223983 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d956522-a44e-4398-a073-458d4a84571f None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:35.226596 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4ef03-8977-4e06-8527-7db0331fa047 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:35.236929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-5d956522-a44e-4398-a073-458d4a84571f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 14:35:35.236929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:35.236929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 14:35:35.236929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 14:35:35.236929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 14:35:35.236929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 14:35:35.236929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 14:35:35.236929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:35.236929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 14:35:35.236929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:35.236929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 14:35:35.236929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:35.236929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:35.236929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:35.236929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:35.236929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:35.236929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:35.238171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:35.238171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:35.238171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:35.238171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:35.238171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:35.238171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:35.238171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 14:35:35.238171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 14:35:35.238171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 14:35:35.238171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 14:35:35.238171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:35.238171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:35.238171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:35.238171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:35.238171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:35.238171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:35.239255 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:35.239255 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:35.239255 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:35.239255 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:35.239255 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:35.239255 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:35.239255 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:35.239255 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:35.239255 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:35.239255 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:35.239255 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 14:35:35.239255 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:35.239255 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 14:35:35.239255 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:35.243621 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 518/1038] 10.222.0.15 () {56 vars in 1105 bytes} [Wed Jul 20 14:35:35 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:35:35.244514 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d956522-a44e-4398-a073-458d4a84571f None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:35.267288 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0327d9-161e-4df6-a51f-888d6144b572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:35.268120 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0327d9-161e-4df6-a51f-888d6144b572 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:35.268540 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0327d9-161e-4df6-a51f-888d6144b572 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:35.268819 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0327d9-161e-4df6-a51f-888d6144b572 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:35.275734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0327d9-161e-4df6-a51f-888d6144b572 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:35.277211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0327d9-161e-4df6-a51f-888d6144b572 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:35.295180 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-1c0327d9-161e-4df6-a51f-888d6144b572 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:35:35.295180 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:35.295180 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:35.295180 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:35.295180 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:35.295180 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:35.295180 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:35.295180 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:35.295180 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:35.295180 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:35.295180 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:35.295180 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:35.295180 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:35:35.295180 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:35:35.295180 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:35:35.296154 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:35:35.296154 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:35.296154 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:35.296154 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:35.296154 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:35.296154 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:35.296154 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:35.296154 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:35.296154 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:35.296154 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:35.296154 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:35.296154 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:35.296154 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:35.296154 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:35.296154 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:35.296154 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:35.296154 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:35.297370 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:35:35.297370 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:35:35.297370 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:35:35.297370 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:35:35.297370 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:35:35.297370 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:35:35.297370 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:35:35.297370 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:35.299198 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 521/1039] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:35:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:35.302848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0327d9-161e-4df6-a51f-888d6144b572 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:35.338572 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:35.354070 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d956522-a44e-4398-a073-458d4a84571f None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:35.390848 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a7af7-11cb-4ce2-92f8-51de19c35963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:35.391260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a7af7-11cb-4ce2-92f8-51de19c35963 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:35.391588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a7af7-11cb-4ce2-92f8-51de19c35963 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:35.391869 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a7af7-11cb-4ce2-92f8-51de19c35963 None admin] PATH_INFO: `/v3/services` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:35.392601 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a7af7-11cb-4ce2-92f8-51de19c35963 None admin] RBAC: Adding query filter params (name=image) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:35.392910 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a7af7-11cb-4ce2-92f8-51de19c35963 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:35.394278 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a7af7-11cb-4ce2-92f8-51de19c35963 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:35.404158 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 519/1040] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:35.405192 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a7af7-11cb-4ce2-92f8-51de19c35963 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:35.412252 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0327d9-161e-4df6-a51f-888d6144b572 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:35.449523 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d250ed-f0ba-4670-9184-62e155446658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:35.450184 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d250ed-f0ba-4670-9184-62e155446658 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:35.450444 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d250ed-f0ba-4670-9184-62e155446658 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:35.450725 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d250ed-f0ba-4670-9184-62e155446658 None admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:35.451549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d250ed-f0ba-4670-9184-62e155446658 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:35.451861 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d250ed-f0ba-4670-9184-62e155446658 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:35.453310 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d250ed-f0ba-4670-9184-62e155446658 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:35.459907 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:35.467117 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 522/1041] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:35:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:35.471618 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d250ed-f0ba-4670-9184-62e155446658 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:35.523150 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a7af7-11cb-4ce2-92f8-51de19c35963 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:35.562090 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf7a74-f57b-4e9c-98a2-1c41f79c9216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:35.563108 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf7a74-f57b-4e9c-98a2-1c41f79c9216 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:35.563464 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf7a74-f57b-4e9c-98a2-1c41f79c9216 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:35.563812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf7a74-f57b-4e9c-98a2-1c41f79c9216 None admin] PATH_INFO: `/v3/services` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:35.564583 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf7a74-f57b-4e9c-98a2-1c41f79c9216 None admin] RBAC: Adding query filter params (type=image) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:35.564963 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf7a74-f57b-4e9c-98a2-1c41f79c9216 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:35.567091 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf7a74-f57b-4e9c-98a2-1c41f79c9216 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:35.572057 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:35.578716 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 520/1042] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:35 2022] GET /identity/v3/services?type=image => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:35.580261 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d250ed-f0ba-4670-9184-62e155446658 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:35.580386 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf7a74-f57b-4e9c-98a2-1c41f79c9216 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:35.624459 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cbf21d-d46a-4260-ac67-22c660811562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:35.625424 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbf21d-d46a-4260-ac67-22c660811562 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:35.625714 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbf21d-d46a-4260-ac67-22c660811562 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:35.626100 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbf21d-d46a-4260-ac67-22c660811562 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:35.626948 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cbf21d-d46a-4260-ac67-22c660811562 None admin] RBAC: Adding query filter params (role.id=59c1c6b5011a4faea5f02d1961352d67, scope.project.id=59333e4f593346e184877d6bd9d22026, user.id=5d4a107ed04348b59d20d1703dbe04eb) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:35.627293 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cbf21d-d46a-4260-ac67-22c660811562 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:35.629686 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cbf21d-d46a-4260-ac67-22c660811562 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:35.639216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 523/1043] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:35:35 2022] GET /identity/v3/role_assignments?user.id=5d4a107ed04348b59d20d1703dbe04eb&scope.project.id=59333e4f593346e184877d6bd9d22026&role.id=59c1c6b5011a4faea5f02d1961352d67 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:35.711651 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf7a74-f57b-4e9c-98a2-1c41f79c9216 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:35.727816 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:35.762060 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458aefb5-ee5e-4cf5-8c39-849c94d9f823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:35.762764 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458aefb5-ee5e-4cf5-8c39-849c94d9f823 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:35.770040 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458aefb5-ee5e-4cf5-8c39-849c94d9f823 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:35.770040 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458aefb5-ee5e-4cf5-8c39-849c94d9f823 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:35.770040 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458aefb5-ee5e-4cf5-8c39-849c94d9f823 None admin] RBAC: Adding query filter params (interface=public, service_id=a8e9a938c7344ec99e56d13b708baa57, region_id=RegionOne) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:35.770040 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458aefb5-ee5e-4cf5-8c39-849c94d9f823 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:35.770040 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458aefb5-ee5e-4cf5-8c39-849c94d9f823 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:35.775035 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 521/1044] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Jul 20 14:35:35 2022] GET /identity/v3/endpoints?service_id=a8e9a938c7344ec99e56d13b708baa57&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:35.784287 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cbf21d-d46a-4260-ac67-22c660811562 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:35.905137 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cbf21d-d46a-4260-ac67-22c660811562 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:35.943277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2981a21-0cb7-4f4a-add5-41c1b18450ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:35.944239 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2981a21-0cb7-4f4a-add5-41c1b18450ba None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:35.944672 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2981a21-0cb7-4f4a-add5-41c1b18450ba None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:35.945105 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2981a21-0cb7-4f4a-add5-41c1b18450ba None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:35.945961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2981a21-0cb7-4f4a-add5-41c1b18450ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:35.947522 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2981a21-0cb7-4f4a-add5-41c1b18450ba None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:35.955204 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 524/1045] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Jul 20 14:35:35 2022] GET /identity/v3/services => generated 1957 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:35:36.771366 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68665e21-c63c-47dd-9cff-5b9a7a300845 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:36.771867 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68665e21-c63c-47dd-9cff-5b9a7a300845 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:36.771867 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68665e21-c63c-47dd-9cff-5b9a7a300845 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:36.773381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 522/1046] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:36.778783 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732db0c-c6ef-4d50-a39e-08f779e07ae3 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:36.779248 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732db0c-c6ef-4d50-a39e-08f779e07ae3 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:36.779349 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732db0c-c6ef-4d50-a39e-08f779e07ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:36.796182 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-2732db0c-c6ef-4d50-a39e-08f779e07ae3 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:36.797676 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-2732db0c-c6ef-4d50-a39e-08f779e07ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:36.815092 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:36.867226 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 525/1047] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:36 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:35:36.887834 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68665e21-c63c-47dd-9cff-5b9a7a300845 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:36.994958 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68665e21-c63c-47dd-9cff-5b9a7a300845 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:37.030640 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78508cc6-4f8f-4646-8203-abe8b4d3a40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:37.031403 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78508cc6-4f8f-4646-8203-abe8b4d3a40a None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:37.031670 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78508cc6-4f8f-4646-8203-abe8b4d3a40a None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:37.032015 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78508cc6-4f8f-4646-8203-abe8b4d3a40a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:37.042761 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78508cc6-4f8f-4646-8203-abe8b4d3a40a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:35:37.050892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-78508cc6-4f8f-4646-8203-abe8b4d3a40a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:35:37.050892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:37.050892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:37.050892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:37.050892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:37.050892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:37.050892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:37.050892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:37.050892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:37.050892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:37.050892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:37.050892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:37.050892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:35:37.050892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:35:37.050892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:35:37.050892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:35:37.052000 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:35:37.052000 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:35:37.052000 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:35:37.052000 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:35:37.052000 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:37.052000 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:37.052000 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:37.052000 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:37.052000 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:37.052000 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:37.052000 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:37.052000 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:37.052000 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:37.052000 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:37.052000 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:37.052000 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:37.058187 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:37.058187 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:37.058187 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:37.058187 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:37.058187 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:35:37.058187 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:35:37.058187 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:35:37.058187 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:35:37.058187 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:35:37.058187 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:35:37.058187 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:35:37.058187 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:37.058187 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 523/1048] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:35:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:37.060119 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2732db0c-c6ef-4d50-a39e-08f779e07ae3 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:37.171617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2732db0c-c6ef-4d50-a39e-08f779e07ae3 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:37.206078 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e850cec-ce4d-46da-a427-bd8aafa4643c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:37.206907 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e850cec-ce4d-46da-a427-bd8aafa4643c None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:37.207219 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e850cec-ce4d-46da-a427-bd8aafa4643c None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:37.207500 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e850cec-ce4d-46da-a427-bd8aafa4643c None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:37.208469 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e850cec-ce4d-46da-a427-bd8aafa4643c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:37.208770 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e850cec-ce4d-46da-a427-bd8aafa4643c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:37.210175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e850cec-ce4d-46da-a427-bd8aafa4643c None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:37.223715 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 526/1049] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:35:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:37.225128 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78508cc6-4f8f-4646-8203-abe8b4d3a40a None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:37.261720 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:37.330698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78508cc6-4f8f-4646-8203-abe8b4d3a40a None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:37.365866 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01e87c-9b32-487c-987c-c8bbca1b08ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:37.366751 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01e87c-9b32-487c-987c-c8bbca1b08ab None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:37.367032 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01e87c-9b32-487c-987c-c8bbca1b08ab None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:37.367308 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01e87c-9b32-487c-987c-c8bbca1b08ab None admin] PATH_INFO: `/v3/users/d9cad1d7f67847e9814a17c86f4809c5` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:37.381579 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe01e87c-9b32-487c-987c-c8bbca1b08ab None admin] RBAC: Authorizing `identity:get_user(user_id=d9cad1d7f67847e9814a17c86f4809c5)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:37.383035 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe01e87c-9b32-487c-987c-c8bbca1b08ab None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:37.389344 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 524/1050] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:35:37 2022] GET /identity/v3/users/d9cad1d7f67847e9814a17c86f4809c5 => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:37.390583 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e850cec-ce4d-46da-a427-bd8aafa4643c None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:37.409916 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:37.500184 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e850cec-ce4d-46da-a427-bd8aafa4643c None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:37.534570 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fde7c7-9d5a-425d-aac0-382c11e85433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:37.535615 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fde7c7-9d5a-425d-aac0-382c11e85433 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:37.536110 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fde7c7-9d5a-425d-aac0-382c11e85433 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:37.536632 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fde7c7-9d5a-425d-aac0-382c11e85433 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:37.605301 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fde7c7-9d5a-425d-aac0-382c11e85433 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:37.607131 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fde7c7-9d5a-425d-aac0-382c11e85433 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:37.625061 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:37.691249 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88fef4-8381-4496-81ee-7ac8d72966c7 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:37.691584 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88fef4-8381-4496-81ee-7ac8d72966c7 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:37.691823 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88fef4-8381-4496-81ee-7ac8d72966c7 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:37.693268 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 525/1051] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:37.698181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7937b17-886e-485e-8422-a77052463a93 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:37.698468 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7937b17-886e-485e-8422-a77052463a93 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:37.698754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7937b17-886e-485e-8422-a77052463a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:37.713294 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 527/1052] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:35:37 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 14:35:37.715232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-c7937b17-886e-485e-8422-a77052463a93 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:37.716444 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-c7937b17-886e-485e-8422-a77052463a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:37.718793 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fde7c7-9d5a-425d-aac0-382c11e85433 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:37.786015 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 526/1053] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:37 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 14:35:37.809118 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7937b17-886e-485e-8422-a77052463a93 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:37.825326 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fde7c7-9d5a-425d-aac0-382c11e85433 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:37.834423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:37.844736 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:37.868597 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ecf139-c732-46c2-814b-49f5c7956615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:37.870200 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ecf139-c732-46c2-814b-49f5c7956615 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:37.870904 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ecf139-c732-46c2-814b-49f5c7956615 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:37.871368 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ecf139-c732-46c2-814b-49f5c7956615 None admin] PATH_INFO: `/v3/projects/808b86c407124d6e89dd8808a540c069` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:37.873283 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ecf139-c732-46c2-814b-49f5c7956615 None admin] RBAC: Authorizing `identity:get_project(project_id=808b86c407124d6e89dd8808a540c069)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:37.875439 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ecf139-c732-46c2-814b-49f5c7956615 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:37.877848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 528/1054] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:35:37 2022] GET /identity/v3/projects/808b86c407124d6e89dd8808a540c069 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:37.884059 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ecf139-c732-46c2-814b-49f5c7956615 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:37.916475 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7937b17-886e-485e-8422-a77052463a93 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:37.955560 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e08a37-a968-4eba-9d60-75d414602186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:37.956397 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e08a37-a968-4eba-9d60-75d414602186 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:37.956679 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e08a37-a968-4eba-9d60-75d414602186 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:37.956959 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e08a37-a968-4eba-9d60-75d414602186 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:37.968146 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e08a37-a968-4eba-9d60-75d414602186 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:35:37.976441 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-44e08a37-a968-4eba-9d60-75d414602186 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 14:35:37.976441 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:37.976441 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:37.976441 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:37.976441 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:37.976441 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:37.976441 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:37.976441 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:37.976441 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:37.976441 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:37.976441 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:37.976441 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:37.976441 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:35:37.976441 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:35:37.976441 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:35:37.976441 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:35:37.977539 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:35:37.977539 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:35:37.977539 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:35:37.977539 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:35:37.977539 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:37.977539 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:37.977539 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:37.977539 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:37.977539 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:37.977539 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:37.977539 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:37.977539 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:37.977539 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:37.977539 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:37.977539 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:37.977539 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:37.978405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:37.978405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:37.978405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:37.978405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:37.978405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:35:37.978405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:35:37.978405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:35:37.978405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:35:37.978405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:35:37.978405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:35:37.978405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 14:35:37.978405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:37.978405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 527/1055] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Jul 20 14:35:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:35:37.983495 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e08a37-a968-4eba-9d60-75d414602186 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:37.993298 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ecf139-c732-46c2-814b-49f5c7956615 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:38.027948 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b308889-299d-43ac-bb55-361f72f28483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:38.028628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b308889-299d-43ac-bb55-361f72f28483 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:38.028910 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b308889-299d-43ac-bb55-361f72f28483 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:38.029186 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b308889-299d-43ac-bb55-361f72f28483 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:38.030040 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b308889-299d-43ac-bb55-361f72f28483 None admin] RBAC: Adding query filter params (role.id=49bd5abf255048a0abaa9538e309f516, scope.project.id=808b86c407124d6e89dd8808a540c069, user.id=d9cad1d7f67847e9814a17c86f4809c5) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:38.030331 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b308889-299d-43ac-bb55-361f72f28483 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:38.031799 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b308889-299d-43ac-bb55-361f72f28483 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:38.037292 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 529/1056] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:35:37 2022] GET /identity/v3/role_assignments?user.id=d9cad1d7f67847e9814a17c86f4809c5&scope.project.id=808b86c407124d6e89dd8808a540c069&role.id=49bd5abf255048a0abaa9538e309f516 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:38.038495 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd2ea6-d5fb-46d2-9686-aa178bf59643 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:38.038862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd2ea6-d5fb-46d2-9686-aa178bf59643 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:38.039079 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd2ea6-d5fb-46d2-9686-aa178bf59643 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:38.040402 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 530/1057] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:38.049243 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f3d4b-551d-450a-91e1-94aa985052d5 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:38.049583 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f3d4b-551d-450a-91e1-94aa985052d5 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:38.049789 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f3d4b-551d-450a-91e1-94aa985052d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:38.068833 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-d58f3d4b-551d-450a-91e1-94aa985052d5 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:38.070092 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-d58f3d4b-551d-450a-91e1-94aa985052d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:38.077141 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:38.097404 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e08a37-a968-4eba-9d60-75d414602186 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:38.120954 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:38.139465 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a65f0-5d45-4f4e-8f22-62e3ff333b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:38.140155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a65f0-5d45-4f4e-8f22-62e3ff333b6e None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:38.140437 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a65f0-5d45-4f4e-8f22-62e3ff333b6e None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:38.140707 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a65f0-5d45-4f4e-8f22-62e3ff333b6e None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:38.141465 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60a65f0-5d45-4f4e-8f22-62e3ff333b6e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:38.141777 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60a65f0-5d45-4f4e-8f22-62e3ff333b6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:38.143192 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60a65f0-5d45-4f4e-8f22-62e3ff333b6e None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:38.147825 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 531/1058] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:38 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 14:35:38.157357 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 528/1059] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Jul 20 14:35:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:38.159345 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f3d4b-551d-450a-91e1-94aa985052d5 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:38.172279 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a65f0-5d45-4f4e-8f22-62e3ff333b6e None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:38.278491 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f3d4b-551d-450a-91e1-94aa985052d5 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:38.279417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a65f0-5d45-4f4e-8f22-62e3ff333b6e None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:38.315127 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b636a9-1fba-40be-9427-0e34db686b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:38.316015 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b636a9-1fba-40be-9427-0e34db686b84 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:38.316127 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b856730-01ad-4c3c-9515-529b8b514c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:38.316330 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b636a9-1fba-40be-9427-0e34db686b84 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:38.316580 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b636a9-1fba-40be-9427-0e34db686b84 None admin] PATH_INFO: `/v3/services/image` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:38.316871 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b856730-01ad-4c3c-9515-529b8b514c92 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:38.317153 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b856730-01ad-4c3c-9515-529b8b514c92 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:38.317246 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b636a9-1fba-40be-9427-0e34db686b84 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:38.317425 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b856730-01ad-4c3c-9515-529b8b514c92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:38.318768 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b636a9-1fba-40be-9427-0e34db686b84 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:38.327976 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b856730-01ad-4c3c-9515-529b8b514c92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:38.329425 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b856730-01ad-4c3c-9515-529b8b514c92 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:38.331540 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-c8b636a9-1fba-40be-9427-0e34db686b84 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 14:35:38.331540 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:38.331540 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 14:35:38.331540 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 14:35:38.331540 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 14:35:38.331540 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 14:35:38.331540 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 14:35:38.331540 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:38.331540 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 14:35:38.331540 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:38.331540 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 14:35:38.331540 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:38.331540 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:38.331540 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:38.331540 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:38.331540 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:38.331540 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:38.332531 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:38.332531 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:38.332531 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:38.332531 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:38.332531 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:38.332531 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:38.332531 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 14:35:38.332531 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 14:35:38.332531 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 14:35:38.332531 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 14:35:38.332531 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:38.332531 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:38.332531 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:38.332531 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:38.332531 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:38.332531 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:38.333469 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:38.333469 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:38.333469 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:38.333469 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:38.333469 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:38.333469 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:38.333469 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:38.333469 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:38.333469 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:38.333469 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:38.333469 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 14:35:38.333469 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:35:38.333469 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 14:35:38.333469 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:38.333469 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 529/1060] 10.222.0.15 () {56 vars in 1105 bytes} [Wed Jul 20 14:35:38 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:38.338923 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b636a9-1fba-40be-9427-0e34db686b84 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:38.350327 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-6b856730-01ad-4c3c-9515-529b8b514c92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:38.350327 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:38.350327 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:38.350327 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:38.350327 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:38.350327 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:38.350327 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:38.350327 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:38.350327 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:38.350327 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:38.350327 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:38.350327 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:38.350327 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:35:38.350327 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:35:38.350327 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:35:38.351383 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:35:38.351383 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:38.351383 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:38.351383 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:38.351383 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:38.351383 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:38.351383 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:38.351383 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:38.351383 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:38.351383 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:38.351383 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:38.351383 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:38.351383 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:38.351383 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:38.351383 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:38.351383 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:38.351383 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:38.352577 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:35:38.352577 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:35:38.352577 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:38.352577 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:38.354798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:38.356403 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 532/1061] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:38.357688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b856730-01ad-4c3c-9515-529b8b514c92 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:38.393411 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:38.455959 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b636a9-1fba-40be-9427-0e34db686b84 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:38.475078 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b856730-01ad-4c3c-9515-529b8b514c92 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:38.500163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf703ef-21d0-4b9e-bd84-95f923de2663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:38.500836 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf703ef-21d0-4b9e-bd84-95f923de2663 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:38.501543 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf703ef-21d0-4b9e-bd84-95f923de2663 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:38.501826 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf703ef-21d0-4b9e-bd84-95f923de2663 None admin] PATH_INFO: `/v3/services` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:38.502532 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf703ef-21d0-4b9e-bd84-95f923de2663 None admin] RBAC: Adding query filter params (name=image) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:38.502846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf703ef-21d0-4b9e-bd84-95f923de2663 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:38.504258 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf703ef-21d0-4b9e-bd84-95f923de2663 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:38.510039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 530/1062] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:38.515508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf703ef-21d0-4b9e-bd84-95f923de2663 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:38.516371 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44850bb3-c290-4881-a0a6-215d6bda14ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:38.517054 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44850bb3-c290-4881-a0a6-215d6bda14ed None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:38.517341 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44850bb3-c290-4881-a0a6-215d6bda14ed None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:38.517620 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44850bb3-c290-4881-a0a6-215d6bda14ed None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:38.518481 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44850bb3-c290-4881-a0a6-215d6bda14ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:38.518654 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44850bb3-c290-4881-a0a6-215d6bda14ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:38.520393 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44850bb3-c290-4881-a0a6-215d6bda14ed None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:38.532391 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 533/1063] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:38.540524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44850bb3-c290-4881-a0a6-215d6bda14ed None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:38.596132 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:38.624164 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf703ef-21d0-4b9e-bd84-95f923de2663 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:38.637218 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:38.647508 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44850bb3-c290-4881-a0a6-215d6bda14ed None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:38.660385 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed2832a-3008-4d75-b42a-4973dcb30bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:38.661061 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2832a-3008-4d75-b42a-4973dcb30bcd None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:38.661365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2832a-3008-4d75-b42a-4973dcb30bcd None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:38.661687 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2832a-3008-4d75-b42a-4973dcb30bcd None admin] PATH_INFO: `/v3/services` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:38.662395 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed2832a-3008-4d75-b42a-4973dcb30bcd None admin] RBAC: Adding query filter params (type=image) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:38.662703 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed2832a-3008-4d75-b42a-4973dcb30bcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:38.664346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed2832a-3008-4d75-b42a-4973dcb30bcd None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:38.669732 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 531/1064] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:38 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:38.675506 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed2832a-3008-4d75-b42a-4973dcb30bcd None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:38.684357 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47f7a3-4114-4b11-b7dd-108ebd7c932b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:38.685167 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47f7a3-4114-4b11-b7dd-108ebd7c932b None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:38.685452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47f7a3-4114-4b11-b7dd-108ebd7c932b None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:38.685785 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47f7a3-4114-4b11-b7dd-108ebd7c932b None admin] PATH_INFO: `/v3/users/nova` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:38.693917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47f7a3-4114-4b11-b7dd-108ebd7c932b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:38.696156 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47f7a3-4114-4b11-b7dd-108ebd7c932b None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:38.710823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-2a47f7a3-4114-4b11-b7dd-108ebd7c932b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 14:35:38.710823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:38.710823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:38.710823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:38.710823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:38.710823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:38.710823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:38.710823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:38.710823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:38.710823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:38.710823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:38.710823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:38.710823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:35:38.710823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:35:38.710823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:35:38.710823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:35:38.711973 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:38.711973 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:38.711973 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:35:38.711973 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:35:38.711973 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:35:38.711973 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:35:38.711973 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:38.711973 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:38.711973 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:38.711973 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:38.711973 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:38.711973 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:38.711973 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:38.711973 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:38.711973 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:38.711973 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:38.711973 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:38.711973 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:38.713004 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:38.713004 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:38.713004 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:35:38.713004 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:35:38.713004 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:35:38.713004 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:35:38.713004 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:35:38.713004 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:35:38.713004 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 14:35:38.713004 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:38.713004 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 534/1065] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Jul 20 14:35:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:38.717786 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47f7a3-4114-4b11-b7dd-108ebd7c932b None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:38.785589 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed2832a-3008-4d75-b42a-4973dcb30bcd None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:38.824064 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753ea70-b3c6-41e5-9bc2-d268a0b4067f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:38.824693 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47f7a3-4114-4b11-b7dd-108ebd7c932b None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:38.824781 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753ea70-b3c6-41e5-9bc2-d268a0b4067f None admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:38.825050 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753ea70-b3c6-41e5-9bc2-d268a0b4067f None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:38.825258 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753ea70-b3c6-41e5-9bc2-d268a0b4067f None admin] PATH_INFO: `/v3/endpoints` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:38.826155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e753ea70-b3c6-41e5-9bc2-d268a0b4067f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:38.828296 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e753ea70-b3c6-41e5-9bc2-d268a0b4067f None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:38.836088 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:38.850003 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 532/1066] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Jul 20 14:35:38 2022] POST /identity/v3/endpoints => generated 327 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 14:35:38.878305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348c4c4-11ef-47b9-a378-85067e4e8663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:38.879153 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348c4c4-11ef-47b9-a378-85067e4e8663 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:38.879383 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348c4c4-11ef-47b9-a378-85067e4e8663 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:38.879667 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348c4c4-11ef-47b9-a378-85067e4e8663 None admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:38.880677 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c348c4c4-11ef-47b9-a378-85067e4e8663 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:38.880998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c348c4c4-11ef-47b9-a378-85067e4e8663 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:38.882509 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c348c4c4-11ef-47b9-a378-85067e4e8663 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:38.906553 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 535/1067] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Jul 20 14:35:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:38.912681 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753ea70-b3c6-41e5-9bc2-d268a0b4067f None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:39.019483 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753ea70-b3c6-41e5-9bc2-d268a0b4067f None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:39.057140 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d2b6cc-a9f1-40a1-abae-0dfc4a5d01b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:39.058005 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d2b6cc-a9f1-40a1-abae-0dfc4a5d01b7 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:39.058292 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d2b6cc-a9f1-40a1-abae-0dfc4a5d01b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:39.058568 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d2b6cc-a9f1-40a1-abae-0dfc4a5d01b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:39.070877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d2b6cc-a9f1-40a1-abae-0dfc4a5d01b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:39.072414 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d2b6cc-a9f1-40a1-abae-0dfc4a5d01b7 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:39.090895 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-d7d2b6cc-a9f1-40a1-abae-0dfc4a5d01b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:39.090895 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:39.090895 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:39.090895 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:39.090895 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:39.090895 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:39.090895 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:39.090895 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:39.090895 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:39.090895 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:39.090895 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:39.090895 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:39.090895 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:35:39.090895 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:35:39.090895 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:35:39.092063 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:35:39.092063 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:39.092063 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:39.092063 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:39.092063 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:39.092063 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:39.092063 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:39.092063 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:39.092063 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:39.092063 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:39.092063 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:39.092063 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:39.092063 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:39.092063 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:39.092063 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:39.092063 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:39.092063 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:39.092989 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:35:39.092989 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:35:39.092989 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:39.092989 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:39.092989 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 533/1068] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:39.099493 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348c4c4-11ef-47b9-a378-85067e4e8663 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:39.104977 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:39.214094 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348c4c4-11ef-47b9-a378-85067e4e8663 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:39.249370 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d5b66-e071-44c6-bafa-fcf80210291e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:39.250076 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d5b66-e071-44c6-bafa-fcf80210291e None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:39.250351 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d5b66-e071-44c6-bafa-fcf80210291e None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:39.250632 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d5b66-e071-44c6-bafa-fcf80210291e None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:39.251394 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d5b66-e071-44c6-bafa-fcf80210291e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:39.251708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d5b66-e071-44c6-bafa-fcf80210291e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:39.253100 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d5b66-e071-44c6-bafa-fcf80210291e None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:39.265173 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 536/1069] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:39.270855 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d2b6cc-a9f1-40a1-abae-0dfc4a5d01b7 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:39.321316 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:39.382141 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d2b6cc-a9f1-40a1-abae-0dfc4a5d01b7 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:39.420929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac34dc-7f51-480e-9be5-029ca49367d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:39.422086 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac34dc-7f51-480e-9be5-029ca49367d0 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:39.422369 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac34dc-7f51-480e-9be5-029ca49367d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:39.422636 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac34dc-7f51-480e-9be5-029ca49367d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:39.429470 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cac34dc-7f51-480e-9be5-029ca49367d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:39.430928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cac34dc-7f51-480e-9be5-029ca49367d0 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:39.445079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-0cac34dc-7f51-480e-9be5-029ca49367d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:35:39.445079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:39.445079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:39.445079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:39.445079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:39.445079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:39.445079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:39.445079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:39.445079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:39.445079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:39.445079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:39.445079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:39.445079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:35:39.445079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:35:39.445079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:35:39.446229 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:35:39.446229 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:39.446229 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:39.446229 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:39.446229 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:39.446229 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:39.446229 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:39.446229 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:39.446229 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:39.446229 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:39.446229 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:39.446229 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:39.446229 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:39.446229 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:39.446229 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:39.446229 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:39.446229 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:39.447406 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:35:39.447406 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:35:39.447406 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:35:39.447406 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:35:39.447406 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:35:39.447406 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:35:39.447406 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:35:39.447406 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:39.447406 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 534/1070] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:35:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:39.454324 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d5b66-e071-44c6-bafa-fcf80210291e None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:39.526941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:39.568524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d5b66-e071-44c6-bafa-fcf80210291e None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:39.605294 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce81de-b9f8-4c39-ac5f-bb653b8def98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:39.605972 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce81de-b9f8-4c39-ac5f-bb653b8def98 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:39.606259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce81de-b9f8-4c39-ac5f-bb653b8def98 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:39.606538 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce81de-b9f8-4c39-ac5f-bb653b8def98 None admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:39.607383 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce81de-b9f8-4c39-ac5f-bb653b8def98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:39.607697 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce81de-b9f8-4c39-ac5f-bb653b8def98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:39.609140 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce81de-b9f8-4c39-ac5f-bb653b8def98 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:39.622214 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 537/1071] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:35:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:39.628240 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac34dc-7f51-480e-9be5-029ca49367d0 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:39.732173 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac34dc-7f51-480e-9be5-029ca49367d0 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:39.740646 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:39.768867 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e8e8a-08e1-492d-9da5-5df9ae6edf60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:39.769768 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e8e8a-08e1-492d-9da5-5df9ae6edf60 None admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:39.770050 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e8e8a-08e1-492d-9da5-5df9ae6edf60 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:39.770326 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e8e8a-08e1-492d-9da5-5df9ae6edf60 None admin] PATH_INFO: `/v3/projects/59333e4f593346e184877d6bd9d22026/users/5d4a107ed04348b59d20d1703dbe04eb/roles/59c1c6b5011a4faea5f02d1961352d67` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:39.772641 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e8e8a-08e1-492d-9da5-5df9ae6edf60 None admin] RBAC: Authorizing `identity:create_grant(project_id=59333e4f593346e184877d6bd9d22026, user_id=5d4a107ed04348b59d20d1703dbe04eb, role_id=59c1c6b5011a4faea5f02d1961352d67)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:39.774166 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e8e8a-08e1-492d-9da5-5df9ae6edf60 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:39.788067 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 535/1072] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jul 20 14:35:39 2022] PUT /identity/v3/projects/59333e4f593346e184877d6bd9d22026/users/5d4a107ed04348b59d20d1703dbe04eb/roles/59c1c6b5011a4faea5f02d1961352d67 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:35:40.034082 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5438f97e-927c-4019-b207-6c002b438544 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:40.034625 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5438f97e-927c-4019-b207-6c002b438544 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:40.034625 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5438f97e-927c-4019-b207-6c002b438544 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:40.036186 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 538/1073] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:40.041970 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d44e2-c85a-4fad-b975-f6327aa31fc0 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:40.042433 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d44e2-c85a-4fad-b975-f6327aa31fc0 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:40.042511 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d44e2-c85a-4fad-b975-f6327aa31fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:40.060887 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-e01d44e2-c85a-4fad-b975-f6327aa31fc0 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:40.062096 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-e01d44e2-c85a-4fad-b975-f6327aa31fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:40.133620 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 536/1074] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:35:40.155701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5438f97e-927c-4019-b207-6c002b438544 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:40.262605 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5438f97e-927c-4019-b207-6c002b438544 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:40.286562 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:40.299029 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f348d0b-4c1f-4e02-8285-47c57a31fe8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:40.299869 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f348d0b-4c1f-4e02-8285-47c57a31fe8d None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:40.300153 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f348d0b-4c1f-4e02-8285-47c57a31fe8d None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:40.300426 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f348d0b-4c1f-4e02-8285-47c57a31fe8d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:40.311578 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f348d0b-4c1f-4e02-8285-47c57a31fe8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:35:40.319333 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-9f348d0b-4c1f-4e02-8285-47c57a31fe8d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:35:40.319333 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:40.319333 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:40.319333 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:40.319333 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:40.319333 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:40.319333 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:40.319333 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:40.319333 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:40.319333 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:40.319333 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:40.319333 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:40.319333 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:35:40.319333 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:35:40.319333 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:35:40.319333 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:35:40.320322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:35:40.320322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:35:40.320322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:35:40.320322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:35:40.320322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:40.320322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:40.320322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:40.320322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:40.320322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:40.320322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:40.320322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:40.320322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:40.320322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:40.320322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:40.320322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:40.320322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:40.321241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:40.321241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:40.321241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:40.321241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:40.321241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:35:40.321241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:35:40.321241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:35:40.321241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:35:40.321241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:35:40.321241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:35:40.321241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:35:40.321241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:40.323448 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 539/1075] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:35:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:40.326714 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d44e2-c85a-4fad-b975-f6327aa31fc0 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:40.423153 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:40.430418 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d44e2-c85a-4fad-b975-f6327aa31fc0 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:40.468437 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add3bf34-e6ac-466c-8e55-378f4bf1a322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:40.469126 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3bf34-e6ac-466c-8e55-378f4bf1a322 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:40.469411 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3bf34-e6ac-466c-8e55-378f4bf1a322 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:40.469687 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3bf34-e6ac-466c-8e55-378f4bf1a322 None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:40.470442 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add3bf34-e6ac-466c-8e55-378f4bf1a322 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:40.470748 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add3bf34-e6ac-466c-8e55-378f4bf1a322 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:40.472175 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add3bf34-e6ac-466c-8e55-378f4bf1a322 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:40.491864 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 537/1076] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:35:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:40.493185 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f348d0b-4c1f-4e02-8285-47c57a31fe8d None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:40.603316 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f348d0b-4c1f-4e02-8285-47c57a31fe8d None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:40.639181 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34461faf-fe3d-43a3-bd84-13208a8aeb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:40.640078 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34461faf-fe3d-43a3-bd84-13208a8aeb36 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:40.640361 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34461faf-fe3d-43a3-bd84-13208a8aeb36 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:40.640645 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34461faf-fe3d-43a3-bd84-13208a8aeb36 None admin] PATH_INFO: `/v3/users/d9cad1d7f67847e9814a17c86f4809c5` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:40.641861 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34461faf-fe3d-43a3-bd84-13208a8aeb36 None admin] RBAC: Authorizing `identity:get_user(user_id=d9cad1d7f67847e9814a17c86f4809c5)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:40.643393 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34461faf-fe3d-43a3-bd84-13208a8aeb36 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:40.649760 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 540/1077] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:35:40 2022] GET /identity/v3/users/d9cad1d7f67847e9814a17c86f4809c5 => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:40.651073 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add3bf34-e6ac-466c-8e55-378f4bf1a322 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:40.758730 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add3bf34-e6ac-466c-8e55-378f4bf1a322 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:40.792513 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2c2e29-9007-4a61-a40c-21b64d330770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:40.793370 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c2e29-9007-4a61-a40c-21b64d330770 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:40.793667 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c2e29-9007-4a61-a40c-21b64d330770 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:40.793937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c2e29-9007-4a61-a40c-21b64d330770 None admin] PATH_INFO: `/v3/projects/808b86c407124d6e89dd8808a540c069` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:40.795159 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2c2e29-9007-4a61-a40c-21b64d330770 None admin] RBAC: Authorizing `identity:get_project(project_id=808b86c407124d6e89dd8808a540c069)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:40.796901 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2c2e29-9007-4a61-a40c-21b64d330770 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:40.798762 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 538/1078] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:35:40 2022] GET /identity/v3/projects/808b86c407124d6e89dd8808a540c069 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:40.804806 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34461faf-fe3d-43a3-bd84-13208a8aeb36 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:40.858431 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:40.908698 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34461faf-fe3d-43a3-bd84-13208a8aeb36 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:40.924569 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132369e-0cde-469b-bc85-0a2a4f4fc658 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:40.924955 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132369e-0cde-469b-bc85-0a2a4f4fc658 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:40.925301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132369e-0cde-469b-bc85-0a2a4f4fc658 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:40.926837 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 539/1079] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:40.932135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c7322-4b06-41ed-9554-834252aa1ca1 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:40.932427 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c7322-4b06-41ed-9554-834252aa1ca1 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:40.932702 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c7322-4b06-41ed-9554-834252aa1ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:40.949766 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ab9c4-1e4b-410d-bcf0-ad5bf0af3ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:40.950257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-938c7322-4b06-41ed-9554-834252aa1ca1 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:40.950666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ab9c4-1e4b-410d-bcf0-ad5bf0af3ee6 None admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:40.950949 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ab9c4-1e4b-410d-bcf0-ad5bf0af3ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:40.951257 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ab9c4-1e4b-410d-bcf0-ad5bf0af3ee6 None admin] PATH_INFO: `/v3/projects/808b86c407124d6e89dd8808a540c069/users/d9cad1d7f67847e9814a17c86f4809c5/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:40.952226 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-938c7322-4b06-41ed-9554-834252aa1ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:40.953251 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067ab9c4-1e4b-410d-bcf0-ad5bf0af3ee6 None admin] RBAC: Authorizing `identity:create_grant(project_id=808b86c407124d6e89dd8808a540c069, user_id=d9cad1d7f67847e9814a17c86f4809c5, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:40.954685 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067ab9c4-1e4b-410d-bcf0-ad5bf0af3ee6 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:40.968967 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 541/1080] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jul 20 14:35:40 2022] PUT /identity/v3/projects/808b86c407124d6e89dd8808a540c069/users/d9cad1d7f67847e9814a17c86f4809c5/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:35:40.975369 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:41.035112 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 540/1081] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:35:41.054106 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ab9c4-1e4b-410d-bcf0-ad5bf0af3ee6 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:41.177240 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ab9c4-1e4b-410d-bcf0-ad5bf0af3ee6 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:41.190455 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:41.215196 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c4b10a-9558-455c-afd1-48fa19812a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:41.215861 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4b10a-9558-455c-afd1-48fa19812a48 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:41.216142 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4b10a-9558-455c-afd1-48fa19812a48 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:41.216377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4b10a-9558-455c-afd1-48fa19812a48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:41.284378 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c4b10a-9558-455c-afd1-48fa19812a48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:41.285838 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c4b10a-9558-455c-afd1-48fa19812a48 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:41.386777 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 542/1082] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:35:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 14:35:41.392619 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c7322-4b06-41ed-9554-834252aa1ca1 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:41.507958 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c7322-4b06-41ed-9554-834252aa1ca1 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:41.541893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5035fcf3-7327-470c-b657-6a57060fbb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:41.542684 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035fcf3-7327-470c-b657-6a57060fbb59 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:41.542965 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035fcf3-7327-470c-b657-6a57060fbb59 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:41.543277 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035fcf3-7327-470c-b657-6a57060fbb59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:41.553698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5035fcf3-7327-470c-b657-6a57060fbb59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:41.555148 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5035fcf3-7327-470c-b657-6a57060fbb59 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:41.572139 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-5035fcf3-7327-470c-b657-6a57060fbb59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:41.572139 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:41.572139 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:41.572139 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:41.572139 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:41.572139 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:41.572139 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:41.572139 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:41.572139 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:41.572139 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:41.572139 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:41.572139 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:41.572139 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:35:41.572139 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:35:41.572139 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:35:41.573107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:35:41.573107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:41.573107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:41.573107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:41.573107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:41.573107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:41.573107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:41.573107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:41.573107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:41.573107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:41.573107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:41.573107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:41.573107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:41.573107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:41.573107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:41.573107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:41.573107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:41.574027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:35:41.574027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:35:41.574027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:41.574027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:41.574027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 541/1083] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:41.580319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c4b10a-9558-455c-afd1-48fa19812a48 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:41.595569 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:41.685679 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c4b10a-9558-455c-afd1-48fa19812a48 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:41.721793 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd07cb0-a096-464a-8348-3ebac5c25b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:41.722460 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd07cb0-a096-464a-8348-3ebac5c25b3b None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:41.722741 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd07cb0-a096-464a-8348-3ebac5c25b3b None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:41.722976 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd07cb0-a096-464a-8348-3ebac5c25b3b None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:41.723753 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd07cb0-a096-464a-8348-3ebac5c25b3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:41.724114 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd07cb0-a096-464a-8348-3ebac5c25b3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:41.725513 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd07cb0-a096-464a-8348-3ebac5c25b3b None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:41.737571 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 543/1084] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:41.841562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe0bf9-e8dc-4649-85f6-bcd336368f97 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:41.841974 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe0bf9-e8dc-4649-85f6-bcd336368f97 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:41.842149 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe0bf9-e8dc-4649-85f6-bcd336368f97 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:41.843763 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 542/1085] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:41.849598 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309d712-51c1-4ba0-bbb1-c15444eacd09 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:41.850048 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309d712-51c1-4ba0-bbb1-c15444eacd09 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:41.850434 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309d712-51c1-4ba0-bbb1-c15444eacd09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:41.869539 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-9309d712-51c1-4ba0-bbb1-c15444eacd09 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:41.870955 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-9309d712-51c1-4ba0-bbb1-c15444eacd09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:41.944206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 544/1086] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:35:41.968923 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe0bf9-e8dc-4649-85f6-bcd336368f97 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:41.983183 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:42.077744 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe0bf9-e8dc-4649-85f6-bcd336368f97 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:42.115391 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee6cd3-ec50-43a8-af7d-8f4f746d3b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:42.116223 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee6cd3-ec50-43a8-af7d-8f4f746d3b36 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:42.116505 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee6cd3-ec50-43a8-af7d-8f4f746d3b36 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:42.116785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee6cd3-ec50-43a8-af7d-8f4f746d3b36 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:42.128149 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbee6cd3-ec50-43a8-af7d-8f4f746d3b36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:35:42.136552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-cbee6cd3-ec50-43a8-af7d-8f4f746d3b36 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 14:35:42.136552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:42.136552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:42.136552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:42.136552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:42.136552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:42.136552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:42.136552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:42.136552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:42.136552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:42.136552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:42.136552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:42.136552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:35:42.136552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:35:42.136552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:35:42.136552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:35:42.137591 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:35:42.137591 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:35:42.137591 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:35:42.137591 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:35:42.137591 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:42.137591 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:42.137591 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:42.137591 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:42.137591 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:42.137591 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:42.137591 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:42.137591 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:42.137591 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:42.137591 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:42.137591 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:42.137591 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:42.138435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:42.138435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:42.138435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:42.138435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:42.138435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:35:42.138435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:35:42.138435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:35:42.138435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:35:42.138435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:35:42.138435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:35:42.138435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 14:35:42.138435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:42.138435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 543/1087] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Jul 20 14:35:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:42.144266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9309d712-51c1-4ba0-bbb1-c15444eacd09 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:42.146630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:42.247968 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9309d712-51c1-4ba0-bbb1-c15444eacd09 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:42.284765 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3d13e-2498-453b-9520-213ec5284856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:42.285472 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3d13e-2498-453b-9520-213ec5284856 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:42.285759 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3d13e-2498-453b-9520-213ec5284856 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:42.286038 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3d13e-2498-453b-9520-213ec5284856 None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:42.286792 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c3d13e-2498-453b-9520-213ec5284856 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:42.287126 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c3d13e-2498-453b-9520-213ec5284856 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:42.288509 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c3d13e-2498-453b-9520-213ec5284856 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:42.301409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 545/1088] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Jul 20 14:35:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:42.302986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee6cd3-ec50-43a8-af7d-8f4f746d3b36 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:42.371519 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:42.409579 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee6cd3-ec50-43a8-af7d-8f4f746d3b36 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:42.448167 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc4265-4d5e-4468-9814-3cf43cbc3817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:42.449042 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc4265-4d5e-4468-9814-3cf43cbc3817 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:42.449331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc4265-4d5e-4468-9814-3cf43cbc3817 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:42.449608 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc4265-4d5e-4468-9814-3cf43cbc3817 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:42.460741 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc4265-4d5e-4468-9814-3cf43cbc3817 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:42.462196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc4265-4d5e-4468-9814-3cf43cbc3817 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:42.482080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-26cc4265-4d5e-4468-9814-3cf43cbc3817 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:42.482080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:42.482080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:42.482080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:42.482080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:42.482080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:42.482080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:42.482080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:42.482080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:42.482080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:42.482080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:42.482080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:42.482080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:35:42.482080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:35:42.482080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:35:42.483123 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:35:42.483123 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:42.483123 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:42.483123 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:42.483123 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:42.483123 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:42.483123 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:42.483123 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:42.483123 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:42.483123 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:42.483123 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:42.483123 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:42.483123 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:42.483123 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:42.483123 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:42.483123 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:42.483123 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:42.484070 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:35:42.484070 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:35:42.484070 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:42.484070 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:42.487435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 544/1089] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:42.491500 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3d13e-2498-453b-9520-213ec5284856 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:42.558466 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:42.598102 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3d13e-2498-453b-9520-213ec5284856 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:42.632657 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e51692c-3fa1-414d-b02f-de519fcc74ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:42.633349 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51692c-3fa1-414d-b02f-de519fcc74ca None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:42.633657 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51692c-3fa1-414d-b02f-de519fcc74ca None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:42.633955 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51692c-3fa1-414d-b02f-de519fcc74ca None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:42.634778 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e51692c-3fa1-414d-b02f-de519fcc74ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:42.635110 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e51692c-3fa1-414d-b02f-de519fcc74ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:42.636510 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e51692c-3fa1-414d-b02f-de519fcc74ca None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:42.648461 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 546/1090] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:42.654215 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc4265-4d5e-4468-9814-3cf43cbc3817 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:42.759752 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc4265-4d5e-4468-9814-3cf43cbc3817 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:42.779939 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:42.794272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d97fd5-9059-4217-9755-5892818f941f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:42.795091 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d97fd5-9059-4217-9755-5892818f941f None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:42.795416 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d97fd5-9059-4217-9755-5892818f941f None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:42.795690 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d97fd5-9059-4217-9755-5892818f941f None admin] PATH_INFO: `/v3/users/nova` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:42.803794 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d97fd5-9059-4217-9755-5892818f941f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:42.805252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d97fd5-9059-4217-9755-5892818f941f None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:42.819260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-44d97fd5-9059-4217-9755-5892818f941f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 14:35:42.819260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:42.819260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:42.819260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:42.819260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:42.819260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:42.819260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:42.819260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:42.819260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:42.819260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:42.819260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:42.819260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:42.819260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:35:42.819260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:35:42.819260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:35:42.819260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:35:42.820322 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:42.820322 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:42.820322 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:35:42.820322 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:35:42.820322 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:35:42.820322 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:35:42.820322 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:42.820322 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:42.820322 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:42.820322 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:42.820322 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:42.820322 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:42.820322 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:42.820322 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:42.820322 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:42.820322 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:42.820322 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:42.820322 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:42.821280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:42.821280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:42.821280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:35:42.821280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:35:42.821280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:35:42.821280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:35:42.821280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:35:42.821280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:35:42.821280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 14:35:42.821280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:42.821280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 545/1091] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Jul 20 14:35:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:42.826509 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e51692c-3fa1-414d-b02f-de519fcc74ca None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:42.931212 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e51692c-3fa1-414d-b02f-de519fcc74ca None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:42.953765 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:42.965825 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a74b4-46fa-40a3-aa6d-c01a2e7b577f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:42.966504 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a74b4-46fa-40a3-aa6d-c01a2e7b577f None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:42.966788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a74b4-46fa-40a3-aa6d-c01a2e7b577f None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:42.967077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a74b4-46fa-40a3-aa6d-c01a2e7b577f None admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:42.967932 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a74b4-46fa-40a3-aa6d-c01a2e7b577f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:42.968244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a74b4-46fa-40a3-aa6d-c01a2e7b577f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:42.969681 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a74b4-46fa-40a3-aa6d-c01a2e7b577f None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:42.986643 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 547/1092] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Jul 20 14:35:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:42.992539 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a74b4-46fa-40a3-aa6d-c01a2e7b577f None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:42.992892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559ca55-4d65-4c5d-9b67-422066e47e2c None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:42.993181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559ca55-4d65-4c5d-9b67-422066e47e2c None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:42.993460 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559ca55-4d65-4c5d-9b67-422066e47e2c None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:42.994783 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 546/1093] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:43.000061 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d03a4-c0a8-4fbd-827d-71b36342394c None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:43.000357 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d03a4-c0a8-4fbd-827d-71b36342394c None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:43.000636 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d03a4-c0a8-4fbd-827d-71b36342394c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:43.017394 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-ee5d03a4-c0a8-4fbd-827d-71b36342394c None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:43.018580 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-ee5d03a4-c0a8-4fbd-827d-71b36342394c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:43.085506 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 547/1094] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:35:43.093771 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a74b4-46fa-40a3-aa6d-c01a2e7b577f None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:43.109181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d03a4-c0a8-4fbd-827d-71b36342394c None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:43.128165 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c0679e-1454-40c9-8acc-6f1bedd07910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:43.128860 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c0679e-1454-40c9-8acc-6f1bedd07910 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:43.129146 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c0679e-1454-40c9-8acc-6f1bedd07910 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:43.129428 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c0679e-1454-40c9-8acc-6f1bedd07910 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:43.178848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:43.197488 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c0679e-1454-40c9-8acc-6f1bedd07910 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:43.198864 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c0679e-1454-40c9-8acc-6f1bedd07910 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:43.210043 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d03a4-c0a8-4fbd-827d-71b36342394c None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:43.221324 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:43.245224 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2694a-67dc-44fe-801f-f6d99af1ae8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:43.246038 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2694a-67dc-44fe-801f-f6d99af1ae8c None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:43.246314 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2694a-67dc-44fe-801f-f6d99af1ae8c None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:43.246585 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2694a-67dc-44fe-801f-f6d99af1ae8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:43.256563 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb2694a-67dc-44fe-801f-f6d99af1ae8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:35:43.264294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-ecb2694a-67dc-44fe-801f-f6d99af1ae8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:35:43.264294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:43.264294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:43.264294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:43.264294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:43.264294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:43.264294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:43.264294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:43.264294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:43.264294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:43.264294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:43.264294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:43.264294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:35:43.264294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:35:43.264294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:35:43.264294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:35:43.265263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:35:43.265263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:35:43.265263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:35:43.265263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:35:43.265263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:43.265263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:43.265263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:43.265263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:43.265263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:43.265263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:43.265263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:43.265263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:43.265263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:43.265263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:43.265263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:43.265263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:43.266141 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:43.266141 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:43.266141 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:43.266141 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:43.266141 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:35:43.266141 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:35:43.266141 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:35:43.266141 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:35:43.266141 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:35:43.266141 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:35:43.266141 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:35:43.266141 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:35:43.266141 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 548/1095] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:35:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:35:43.271571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2694a-67dc-44fe-801f-f6d99af1ae8c None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:43.300444 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 548/1096] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:35:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 14:35:43.306254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c0679e-1454-40c9-8acc-6f1bedd07910 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:43.376280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2694a-67dc-44fe-801f-f6d99af1ae8c None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:43.385606 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:43.405213 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c0679e-1454-40c9-8acc-6f1bedd07910 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:43.409544 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a620c67-5c32-468a-860d-005b98c4e6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:43.410218 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a620c67-5c32-468a-860d-005b98c4e6e4 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:43.410751 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a620c67-5c32-468a-860d-005b98c4e6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:43.410751 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a620c67-5c32-468a-860d-005b98c4e6e4 None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:43.411658 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a620c67-5c32-468a-860d-005b98c4e6e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:43.411991 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a620c67-5c32-468a-860d-005b98c4e6e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:43.413404 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a620c67-5c32-468a-860d-005b98c4e6e4 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:43.422134 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 549/1097] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:35:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:43.428069 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a620c67-5c32-468a-860d-005b98c4e6e4 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:43.442188 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b965a-b00f-43f8-921d-13151eb9bf9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:43.443040 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b965a-b00f-43f8-921d-13151eb9bf9e None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:43.443376 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b965a-b00f-43f8-921d-13151eb9bf9e None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:43.443650 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b965a-b00f-43f8-921d-13151eb9bf9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:43.455097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b965a-b00f-43f8-921d-13151eb9bf9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:43.456513 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b965a-b00f-43f8-921d-13151eb9bf9e None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:43.460473 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:43.476743 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-104b965a-b00f-43f8-921d-13151eb9bf9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:43.476743 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:43.476743 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:43.476743 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:43.476743 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:43.476743 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:43.476743 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:43.476743 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:43.476743 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:43.476743 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:43.476743 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:43.476743 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:43.476743 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:35:43.476743 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:35:43.476743 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:35:43.477706 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:35:43.477706 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:43.477706 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:43.477706 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:43.477706 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:43.477706 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:43.477706 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:43.477706 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:43.477706 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:43.477706 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:43.477706 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:43.477706 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:43.477706 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:43.477706 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:43.477706 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:43.477706 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:43.477706 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:43.478605 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:35:43.478605 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:35:43.478605 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:35:43.478605 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:43.478605 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 549/1098] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:35:43.487683 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b965a-b00f-43f8-921d-13151eb9bf9e None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:43.533473 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a620c67-5c32-468a-860d-005b98c4e6e4 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:43.567725 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f69bfa6-21a7-47c0-a183-a068560de939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:43.568609 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69bfa6-21a7-47c0-a183-a068560de939 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:43.568893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69bfa6-21a7-47c0-a183-a068560de939 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:43.569171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69bfa6-21a7-47c0-a183-a068560de939 None admin] PATH_INFO: `/v3/users/d9cad1d7f67847e9814a17c86f4809c5` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:43.570396 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f69bfa6-21a7-47c0-a183-a068560de939 None admin] RBAC: Authorizing `identity:get_user(user_id=d9cad1d7f67847e9814a17c86f4809c5)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:43.571883 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f69bfa6-21a7-47c0-a183-a068560de939 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:43.573540 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 550/1099] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:35:43 2022] GET /identity/v3/users/d9cad1d7f67847e9814a17c86f4809c5 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:43.579388 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f69bfa6-21a7-47c0-a183-a068560de939 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:43.584594 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b965a-b00f-43f8-921d-13151eb9bf9e None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:43.619768 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12a4a3-de03-4d36-b404-aa3bca23ee86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:43.620448 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12a4a3-de03-4d36-b404-aa3bca23ee86 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:43.620734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12a4a3-de03-4d36-b404-aa3bca23ee86 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:43.621041 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12a4a3-de03-4d36-b404-aa3bca23ee86 None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:43.621771 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c12a4a3-de03-4d36-b404-aa3bca23ee86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:43.622079 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c12a4a3-de03-4d36-b404-aa3bca23ee86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:43.623511 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c12a4a3-de03-4d36-b404-aa3bca23ee86 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:43.624971 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:43.635878 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 550/1100] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:35:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:43.641704 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12a4a3-de03-4d36-b404-aa3bca23ee86 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:43.675380 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:43.679030 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f69bfa6-21a7-47c0-a183-a068560de939 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:43.712985 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fccdc6-c913-4c53-ac9a-1c997aabde1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:43.713562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fccdc6-c913-4c53-ac9a-1c997aabde1f None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:43.713850 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fccdc6-c913-4c53-ac9a-1c997aabde1f None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:43.714127 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fccdc6-c913-4c53-ac9a-1c997aabde1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:43.739754 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12a4a3-de03-4d36-b404-aa3bca23ee86 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:43.774885 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41118f46-43dc-4507-8592-7ba6d5aa9e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:43.776000 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41118f46-43dc-4507-8592-7ba6d5aa9e53 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:43.776000 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41118f46-43dc-4507-8592-7ba6d5aa9e53 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:43.776254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41118f46-43dc-4507-8592-7ba6d5aa9e53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:43.783534 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41118f46-43dc-4507-8592-7ba6d5aa9e53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:43.783867 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fccdc6-c913-4c53-ac9a-1c997aabde1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:43.785005 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41118f46-43dc-4507-8592-7ba6d5aa9e53 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:43.785339 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fccdc6-c913-4c53-ac9a-1c997aabde1f None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:43.801452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-41118f46-43dc-4507-8592-7ba6d5aa9e53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:35:43.801452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:35:43.801452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:35:43.801452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:35:43.801452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:35:43.801452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:35:43.801452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:35:43.801452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:35:43.801452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:35:43.801452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:35:43.801452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:35:43.801452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:35:43.801452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:35:43.801452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:35:43.801452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:35:43.802558 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:35:43.802558 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:35:43.802558 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:35:43.802558 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:35:43.802558 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:35:43.802558 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:35:43.802558 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:35:43.802558 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:35:43.802558 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:35:43.802558 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:35:43.802558 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:35:43.802558 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:35:43.802558 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:35:43.802558 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:35:43.802558 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:35:43.802558 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:35:43.802558 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:35:43.803518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:35:43.803518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:35:43.803518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:35:43.803518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:35:43.803518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:35:43.803518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:35:43.803518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:35:43.803518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:35:43.803518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 551/1101] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:35:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:35:43.808788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41118f46-43dc-4507-8592-7ba6d5aa9e53 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:43.864175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:43.882863 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:43.891267 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 551/1102] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:35:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 14:35:43.892974 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fccdc6-c913-4c53-ac9a-1c997aabde1f None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:43.909675 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41118f46-43dc-4507-8592-7ba6d5aa9e53 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:43.945198 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf41d9-5764-4079-b413-7d00f51f5b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:43.945871 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf41d9-5764-4079-b413-7d00f51f5b34 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:43.946157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf41d9-5764-4079-b413-7d00f51f5b34 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:43.946425 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf41d9-5764-4079-b413-7d00f51f5b34 None admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:43.947248 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cf41d9-5764-4079-b413-7d00f51f5b34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:43.947560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cf41d9-5764-4079-b413-7d00f51f5b34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:43.949005 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cf41d9-5764-4079-b413-7d00f51f5b34 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:43.960142 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 552/1103] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:35:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:43.966162 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf41d9-5764-4079-b413-7d00f51f5b34 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:43.990991 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fccdc6-c913-4c53-ac9a-1c997aabde1f None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:44.026042 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95295b3-075e-40df-b543-48a3d1314ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:44.026902 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95295b3-075e-40df-b543-48a3d1314ffd None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:44.027215 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95295b3-075e-40df-b543-48a3d1314ffd None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:44.027489 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95295b3-075e-40df-b543-48a3d1314ffd None admin] PATH_INFO: `/v3/projects/808b86c407124d6e89dd8808a540c069` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:44.028682 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95295b3-075e-40df-b543-48a3d1314ffd None admin] RBAC: Authorizing `identity:get_project(project_id=808b86c407124d6e89dd8808a540c069)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:44.030130 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95295b3-075e-40df-b543-48a3d1314ffd None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:44.037071 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 552/1104] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:35:43 2022] GET /identity/v3/projects/808b86c407124d6e89dd8808a540c069 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:44.038131 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95295b3-075e-40df-b543-48a3d1314ffd None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:44.066579 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf41d9-5764-4079-b413-7d00f51f5b34 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:44.080463 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:44.101585 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0722f1-0782-4300-bd0d-60503ebe44ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:44.102521 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0722f1-0782-4300-bd0d-60503ebe44ab None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:44.102912 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0722f1-0782-4300-bd0d-60503ebe44ab None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:44.103322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0722f1-0782-4300-bd0d-60503ebe44ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:44.104605 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0722f1-0782-4300-bd0d-60503ebe44ab None admin] RBAC: Adding query filter params (role.id=59c1c6b5011a4faea5f02d1961352d67, scope.project.id=59333e4f593346e184877d6bd9d22026, user.id=5d4a107ed04348b59d20d1703dbe04eb) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:44.105082 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0722f1-0782-4300-bd0d-60503ebe44ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:44.105263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:44.106533 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0722f1-0782-4300-bd0d-60503ebe44ab None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:44.113048 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 553/1105] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:35:43 2022] GET /identity/v3/role_assignments?user.id=5d4a107ed04348b59d20d1703dbe04eb&scope.project.id=59333e4f593346e184877d6bd9d22026&role.id=59c1c6b5011a4faea5f02d1961352d67 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:44.143379 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95295b3-075e-40df-b543-48a3d1314ffd None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:44.192144 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2934ae9-db7c-4b37-bb71-fdc12d2b17a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:44.193253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2934ae9-db7c-4b37-bb71-fdc12d2b17a3 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:44.193683 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2934ae9-db7c-4b37-bb71-fdc12d2b17a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:44.194103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2934ae9-db7c-4b37-bb71-fdc12d2b17a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:44.195418 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2934ae9-db7c-4b37-bb71-fdc12d2b17a3 None admin] RBAC: Adding query filter params (role.id=49bd5abf255048a0abaa9538e309f516, scope.project.id=808b86c407124d6e89dd8808a540c069, user.id=d9cad1d7f67847e9814a17c86f4809c5) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:44.195733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2934ae9-db7c-4b37-bb71-fdc12d2b17a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:44.197548 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2934ae9-db7c-4b37-bb71-fdc12d2b17a3 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:44.207037 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 553/1106] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:35:44 2022] GET /identity/v3/role_assignments?user.id=d9cad1d7f67847e9814a17c86f4809c5&scope.project.id=808b86c407124d6e89dd8808a540c069&role.id=49bd5abf255048a0abaa9538e309f516 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:46.303534 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd36055-7d57-477f-847c-c46d45ed6f08 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:46.304388 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd36055-7d57-477f-847c-c46d45ed6f08 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:46.304388 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd36055-7d57-477f-847c-c46d45ed6f08 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:46.306611 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 554/1107] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:46.313037 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f2e40-0ffe-4b9c-997c-1bad6cc3ab34 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:46.313762 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f2e40-0ffe-4b9c-997c-1bad6cc3ab34 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:46.313906 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f2e40-0ffe-4b9c-997c-1bad6cc3ab34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:46.334526 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-d40f2e40-0ffe-4b9c-997c-1bad6cc3ab34 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:46.335767 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-d40f2e40-0ffe-4b9c-997c-1bad6cc3ab34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:46.407919 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 554/1108] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:35:46.448852 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd36055-7d57-477f-847c-c46d45ed6f08 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:46.576259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd36055-7d57-477f-847c-c46d45ed6f08 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:46.615433 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750351b6-221c-4d86-86aa-6e629d80a2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:46.616223 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750351b6-221c-4d86-86aa-6e629d80a2ea None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:46.616500 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750351b6-221c-4d86-86aa-6e629d80a2ea None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:46.616773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750351b6-221c-4d86-86aa-6e629d80a2ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:46.618046 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750351b6-221c-4d86-86aa-6e629d80a2ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:46.619528 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750351b6-221c-4d86-86aa-6e629d80a2ea None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:46.621148 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 555/1109] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:35:46.627257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40f2e40-0ffe-4b9c-997c-1bad6cc3ab34 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:46.694529 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:46.736050 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40f2e40-0ffe-4b9c-997c-1bad6cc3ab34 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:46.771647 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b86934-bd17-4e69-99e5-883456aaa64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:46.772305 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b86934-bd17-4e69-99e5-883456aaa64e None admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:46.772589 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b86934-bd17-4e69-99e5-883456aaa64e None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:46.772864 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b86934-bd17-4e69-99e5-883456aaa64e None admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:46.773572 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b86934-bd17-4e69-99e5-883456aaa64e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:46.774996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b86934-bd17-4e69-99e5-883456aaa64e None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:46.795423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 555/1110] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jul 20 14:35:46 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:35:48.833708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fee03-678d-422a-97c5-afbdbbc1ea59 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:48.834595 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fee03-678d-422a-97c5-afbdbbc1ea59 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:48.834595 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fee03-678d-422a-97c5-afbdbbc1ea59 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:48.836887 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 556/1111] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:48.845743 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138abf3-70fe-43b6-88c2-ceaea97a979a None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:48.846451 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138abf3-70fe-43b6-88c2-ceaea97a979a None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:48.846622 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138abf3-70fe-43b6-88c2-ceaea97a979a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:48.871833 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-2138abf3-70fe-43b6-88c2-ceaea97a979a None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:48.873045 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-2138abf3-70fe-43b6-88c2-ceaea97a979a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:48.942672 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 556/1112] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:35:48.967518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890fee03-678d-422a-97c5-afbdbbc1ea59 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:49.027681 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:49.095351 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890fee03-678d-422a-97c5-afbdbbc1ea59 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:49.131347 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f408fb9-b647-4de3-afdb-b7b8c4dd5c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:49.132251 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f408fb9-b647-4de3-afdb-b7b8c4dd5c09 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:49.132632 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f408fb9-b647-4de3-afdb-b7b8c4dd5c09 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:49.132905 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f408fb9-b647-4de3-afdb-b7b8c4dd5c09 None admin] PATH_INFO: `/v3/roles/b74189174ce24c1382712858dc35f3ce` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:49.134464 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f408fb9-b647-4de3-afdb-b7b8c4dd5c09 None admin] RBAC: Authorizing `identity:get_role(role_id=b74189174ce24c1382712858dc35f3ce)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:49.135980 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f408fb9-b647-4de3-afdb-b7b8c4dd5c09 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:49.137401 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 557/1113] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:35:48 2022] GET /identity/v3/roles/b74189174ce24c1382712858dc35f3ce => generated 223 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:49.143661 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2138abf3-70fe-43b6-88c2-ceaea97a979a None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:49.198020 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:49.274593 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2138abf3-70fe-43b6-88c2-ceaea97a979a None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:49.322456 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa6be8-3f71-4073-89ac-e2a634c37452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:49.323361 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa6be8-3f71-4073-89ac-e2a634c37452 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:49.323638 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa6be8-3f71-4073-89ac-e2a634c37452 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:49.323909 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa6be8-3f71-4073-89ac-e2a634c37452 None admin] PATH_INFO: `/v3/users/0f1c899857b34b8fa36ac4992cb7361d` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:49.340080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fa6be8-3f71-4073-89ac-e2a634c37452 None admin] RBAC: Authorizing `identity:get_user(user_id=0f1c899857b34b8fa36ac4992cb7361d)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:49.341577 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fa6be8-3f71-4073-89ac-e2a634c37452 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:49.343312 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 557/1114] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:35:49 2022] GET /identity/v3/users/0f1c899857b34b8fa36ac4992cb7361d => generated 286 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:49.349755 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f408fb9-b647-4de3-afdb-b7b8c4dd5c09 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:49.466792 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f408fb9-b647-4de3-afdb-b7b8c4dd5c09 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:49.481523 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:49.504510 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ac953-2042-4593-9ec6-3d82090caec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:49.505216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ac953-2042-4593-9ec6-3d82090caec8 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:49.505501 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ac953-2042-4593-9ec6-3d82090caec8 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:49.505780 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ac953-2042-4593-9ec6-3d82090caec8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:49.575343 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888ac953-2042-4593-9ec6-3d82090caec8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:49.576806 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888ac953-2042-4593-9ec6-3d82090caec8 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:49.677643 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 558/1115] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:35:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 14:35:49.684431 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa6be8-3f71-4073-89ac-e2a634c37452 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:49.806911 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa6be8-3f71-4073-89ac-e2a634c37452 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:49.814551 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:49.843402 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f23673-774f-4e93-93e6-d627baec83cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:49.844287 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f23673-774f-4e93-93e6-d627baec83cb None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:49.844574 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f23673-774f-4e93-93e6-d627baec83cb None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:49.844847 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f23673-774f-4e93-93e6-d627baec83cb None admin] PATH_INFO: `/v3/projects/808b86c407124d6e89dd8808a540c069` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:49.846039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f23673-774f-4e93-93e6-d627baec83cb None admin] RBAC: Authorizing `identity:get_project(project_id=808b86c407124d6e89dd8808a540c069)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:49.847522 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f23673-774f-4e93-93e6-d627baec83cb None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:49.849264 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 558/1116] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:35:49 2022] GET /identity/v3/projects/808b86c407124d6e89dd8808a540c069 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:49.855794 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ac953-2042-4593-9ec6-3d82090caec8 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:49.871593 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:49.974720 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ac953-2042-4593-9ec6-3d82090caec8 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:50.021285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2538a49-90d2-43e8-8508-d2ad7ecdcda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:50.021965 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2538a49-90d2-43e8-8508-d2ad7ecdcda3 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:50.022256 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2538a49-90d2-43e8-8508-d2ad7ecdcda3 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:50.022531 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2538a49-90d2-43e8-8508-d2ad7ecdcda3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:50.023452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2538a49-90d2-43e8-8508-d2ad7ecdcda3 None admin] RBAC: Adding query filter params (role.id=b74189174ce24c1382712858dc35f3ce, scope.project.id=808b86c407124d6e89dd8808a540c069, user.id=0f1c899857b34b8fa36ac4992cb7361d) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:50.023731 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2538a49-90d2-43e8-8508-d2ad7ecdcda3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:50.025278 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2538a49-90d2-43e8-8508-d2ad7ecdcda3 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:50.030991 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 559/1117] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:35:49 2022] GET /identity/v3/role_assignments?user.id=0f1c899857b34b8fa36ac4992cb7361d&scope.project.id=808b86c407124d6e89dd8808a540c069&role.id=b74189174ce24c1382712858dc35f3ce => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:52.018088 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2de1d-f8cc-4d78-9b93-79fdd880234c None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:52.018705 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2de1d-f8cc-4d78-9b93-79fdd880234c None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:52.018705 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2de1d-f8cc-4d78-9b93-79fdd880234c None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:52.020092 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 559/1118] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:52.026620 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d2e6d-fb9c-48cb-9f3e-3110460606a9 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:52.027395 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d2e6d-fb9c-48cb-9f3e-3110460606a9 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:52.027526 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d2e6d-fb9c-48cb-9f3e-3110460606a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:52.056377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-6a2d2e6d-fb9c-48cb-9f3e-3110460606a9 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:52.057648 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-6a2d2e6d-fb9c-48cb-9f3e-3110460606a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:52.138464 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 560/1119] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:35:52.163599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c2de1d-f8cc-4d78-9b93-79fdd880234c None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:52.279611 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c2de1d-f8cc-4d78-9b93-79fdd880234c None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:52.320423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7757358-efd9-40e9-9355-4f4ea4464157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:52.321281 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7757358-efd9-40e9-9355-4f4ea4464157 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:52.321560 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7757358-efd9-40e9-9355-4f4ea4464157 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:52.321828 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7757358-efd9-40e9-9355-4f4ea4464157 None admin] PATH_INFO: `/v3/roles/b74189174ce24c1382712858dc35f3ce` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:52.323369 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7757358-efd9-40e9-9355-4f4ea4464157 None admin] RBAC: Authorizing `identity:get_role(role_id=b74189174ce24c1382712858dc35f3ce)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:52.324731 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7757358-efd9-40e9-9355-4f4ea4464157 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:52.326260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 560/1120] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:35:52 2022] GET /identity/v3/roles/b74189174ce24c1382712858dc35f3ce => generated 223 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:52.332031 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d2e6d-fb9c-48cb-9f3e-3110460606a9 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:52.343026 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:52.453562 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d2e6d-fb9c-48cb-9f3e-3110460606a9 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:52.492815 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97526d99-0eb8-4dba-81cc-fbb792d88e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:52.493713 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97526d99-0eb8-4dba-81cc-fbb792d88e66 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:52.493992 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97526d99-0eb8-4dba-81cc-fbb792d88e66 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:52.494264 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97526d99-0eb8-4dba-81cc-fbb792d88e66 None admin] PATH_INFO: `/v3/users/0f1c899857b34b8fa36ac4992cb7361d` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:52.495593 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97526d99-0eb8-4dba-81cc-fbb792d88e66 None admin] RBAC: Authorizing `identity:get_user(user_id=0f1c899857b34b8fa36ac4992cb7361d)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:52.497056 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97526d99-0eb8-4dba-81cc-fbb792d88e66 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:52.498704 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 561/1121] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:35:52 2022] GET /identity/v3/users/0f1c899857b34b8fa36ac4992cb7361d => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:52.505213 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7757358-efd9-40e9-9355-4f4ea4464157 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:52.557300 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:52.628415 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7757358-efd9-40e9-9355-4f4ea4464157 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:52.668835 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22944a35-48e5-49c5-942c-cabc54e62e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:52.669741 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22944a35-48e5-49c5-942c-cabc54e62e74 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:52.670032 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22944a35-48e5-49c5-942c-cabc54e62e74 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:52.670310 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22944a35-48e5-49c5-942c-cabc54e62e74 None admin] PATH_INFO: `/v3/projects/808b86c407124d6e89dd8808a540c069` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:52.672166 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22944a35-48e5-49c5-942c-cabc54e62e74 None admin] RBAC: Authorizing `identity:get_project(project_id=808b86c407124d6e89dd8808a540c069)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:52.674384 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22944a35-48e5-49c5-942c-cabc54e62e74 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:52.677103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 561/1122] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:35:52 2022] GET /identity/v3/projects/808b86c407124d6e89dd8808a540c069 => generated 311 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:52.684256 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97526d99-0eb8-4dba-81cc-fbb792d88e66 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:52.765988 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:52.791134 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97526d99-0eb8-4dba-81cc-fbb792d88e66 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:52.827709 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3d06e-eb70-473c-a117-50b7662244bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:52.828534 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3d06e-eb70-473c-a117-50b7662244bc None admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:52.828821 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3d06e-eb70-473c-a117-50b7662244bc None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:52.829120 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3d06e-eb70-473c-a117-50b7662244bc None admin] PATH_INFO: `/v3/projects/808b86c407124d6e89dd8808a540c069/users/0f1c899857b34b8fa36ac4992cb7361d/roles/b74189174ce24c1382712858dc35f3ce` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:52.831712 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b3d06e-eb70-473c-a117-50b7662244bc None admin] RBAC: Authorizing `identity:create_grant(project_id=808b86c407124d6e89dd8808a540c069, user_id=0f1c899857b34b8fa36ac4992cb7361d, role_id=b74189174ce24c1382712858dc35f3ce)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:52.833248 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b3d06e-eb70-473c-a117-50b7662244bc None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:52.848381 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 562/1123] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jul 20 14:35:52 2022] PUT /identity/v3/projects/808b86c407124d6e89dd8808a540c069/users/0f1c899857b34b8fa36ac4992cb7361d/roles/b74189174ce24c1382712858dc35f3ce => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:35:54.837704 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d415a26-4457-4120-88e3-c14d4a25943d None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:54.838582 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d415a26-4457-4120-88e3-c14d4a25943d None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:54.838582 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d415a26-4457-4120-88e3-c14d4a25943d None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:54.840965 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 562/1124] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:54.849570 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db5f75-9a62-488d-8715-1d7d60faa385 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:54.849980 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db5f75-9a62-488d-8715-1d7d60faa385 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:54.850197 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db5f75-9a62-488d-8715-1d7d60faa385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:54.873831 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-38db5f75-9a62-488d-8715-1d7d60faa385 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:54.875027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-38db5f75-9a62-488d-8715-1d7d60faa385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:54.946056 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 563/1125] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:35:54.969858 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d415a26-4457-4120-88e3-c14d4a25943d None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:55.081513 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d415a26-4457-4120-88e3-c14d4a25943d None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:55.092777 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:55.129173 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7de47-6a97-4f7a-ba8a-9bd4a0b193ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:55.130073 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7de47-6a97-4f7a-ba8a-9bd4a0b193ff None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:55.130357 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7de47-6a97-4f7a-ba8a-9bd4a0b193ff None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:55.130695 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7de47-6a97-4f7a-ba8a-9bd4a0b193ff None admin] PATH_INFO: `/v3/roles/b74189174ce24c1382712858dc35f3ce` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:55.132546 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f7de47-6a97-4f7a-ba8a-9bd4a0b193ff None admin] RBAC: Authorizing `identity:get_role(role_id=b74189174ce24c1382712858dc35f3ce)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:55.133971 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f7de47-6a97-4f7a-ba8a-9bd4a0b193ff None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:55.135432 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 563/1126] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:35:54 2022] GET /identity/v3/roles/b74189174ce24c1382712858dc35f3ce => generated 223 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:55.144719 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db5f75-9a62-488d-8715-1d7d60faa385 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:55.216135 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:55.258517 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db5f75-9a62-488d-8715-1d7d60faa385 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:55.294075 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d378217d-6ab6-448b-bcd7-79c17863a80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:55.294957 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d378217d-6ab6-448b-bcd7-79c17863a80f None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:55.295276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d378217d-6ab6-448b-bcd7-79c17863a80f None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:55.295551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d378217d-6ab6-448b-bcd7-79c17863a80f None admin] PATH_INFO: `/v3/users/0f1c899857b34b8fa36ac4992cb7361d` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:55.297861 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d378217d-6ab6-448b-bcd7-79c17863a80f None admin] RBAC: Authorizing `identity:get_user(user_id=0f1c899857b34b8fa36ac4992cb7361d)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:55.300158 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d378217d-6ab6-448b-bcd7-79c17863a80f None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:55.302694 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 564/1127] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:35:55 2022] GET /identity/v3/users/0f1c899857b34b8fa36ac4992cb7361d => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:55.311390 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7de47-6a97-4f7a-ba8a-9bd4a0b193ff None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:55.424269 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7de47-6a97-4f7a-ba8a-9bd4a0b193ff None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:55.460175 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e8a0c-17dc-4f75-96ef-63ebeb57dcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:55.460820 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e8a0c-17dc-4f75-96ef-63ebeb57dcc0 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:55.461105 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e8a0c-17dc-4f75-96ef-63ebeb57dcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:55.461376 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e8a0c-17dc-4f75-96ef-63ebeb57dcc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:55.508218 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:55.533384 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e8a0c-17dc-4f75-96ef-63ebeb57dcc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:55.534874 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e8a0c-17dc-4f75-96ef-63ebeb57dcc0 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:55.642292 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 564/1128] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:35:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 14:35:55.648666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d378217d-6ab6-448b-bcd7-79c17863a80f None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:55.754462 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d378217d-6ab6-448b-bcd7-79c17863a80f None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:55.791286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad5771-6830-442c-b219-f29d5902821d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:55.792146 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad5771-6830-442c-b219-f29d5902821d None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:55.792432 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad5771-6830-442c-b219-f29d5902821d None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:55.792715 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad5771-6830-442c-b219-f29d5902821d None admin] PATH_INFO: `/v3/projects/808b86c407124d6e89dd8808a540c069` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:55.794093 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad5771-6830-442c-b219-f29d5902821d None admin] RBAC: Authorizing `identity:get_project(project_id=808b86c407124d6e89dd8808a540c069)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:55.795689 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad5771-6830-442c-b219-f29d5902821d None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:55.797748 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 565/1129] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:35:55 2022] GET /identity/v3/projects/808b86c407124d6e89dd8808a540c069 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:55.807688 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e8a0c-17dc-4f75-96ef-63ebeb57dcc0 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:55.898285 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:55.921034 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e8a0c-17dc-4f75-96ef-63ebeb57dcc0 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:55.955991 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ff6c9-c3e0-4ed0-89c4-9e3647d3bc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:55.956639 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ff6c9-c3e0-4ed0-89c4-9e3647d3bc7a None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:55.956914 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ff6c9-c3e0-4ed0-89c4-9e3647d3bc7a None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:55.957186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ff6c9-c3e0-4ed0-89c4-9e3647d3bc7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:55.958037 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ff6c9-c3e0-4ed0-89c4-9e3647d3bc7a None admin] RBAC: Adding query filter params (role.id=b74189174ce24c1382712858dc35f3ce, scope.project.id=808b86c407124d6e89dd8808a540c069, user.id=0f1c899857b34b8fa36ac4992cb7361d) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:35:55.958349 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ff6c9-c3e0-4ed0-89c4-9e3647d3bc7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:55.959801 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ff6c9-c3e0-4ed0-89c4-9e3647d3bc7a None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:55.965667 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 565/1130] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:35:55 2022] GET /identity/v3/role_assignments?user.id=0f1c899857b34b8fa36ac4992cb7361d&scope.project.id=808b86c407124d6e89dd8808a540c069&role.id=b74189174ce24c1382712858dc35f3ce => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:57.973661 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a98d52-e67d-4171-b35b-b1bde601dfed None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:57.974539 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a98d52-e67d-4171-b35b-b1bde601dfed None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:57.974539 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a98d52-e67d-4171-b35b-b1bde601dfed None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:57.977065 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 566/1131] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:35:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:35:57.987075 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8691b8-8a14-42b0-aede-39126d4ab87a None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:57.987818 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8691b8-8a14-42b0-aede-39126d4ab87a None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:57.988143 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8691b8-8a14-42b0-aede-39126d4ab87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:58.008602 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-6c8691b8-8a14-42b0-aede-39126d4ab87a None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:35:58.009829 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-6c8691b8-8a14-42b0-aede-39126d4ab87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:35:58.086842 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 566/1132] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:35:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:35:58.113113 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a98d52-e67d-4171-b35b-b1bde601dfed None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:58.116518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:58.222240 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a98d52-e67d-4171-b35b-b1bde601dfed None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:58.259457 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b435e9-5a1d-4bb9-9741-e75f5914b619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:58.260518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b435e9-5a1d-4bb9-9741-e75f5914b619 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:58.260888 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b435e9-5a1d-4bb9-9741-e75f5914b619 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:58.261250 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b435e9-5a1d-4bb9-9741-e75f5914b619 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:58.263373 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b435e9-5a1d-4bb9-9741-e75f5914b619 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:58.265639 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b435e9-5a1d-4bb9-9741-e75f5914b619 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:58.268179 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 567/1133] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:35:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:35:58.274399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8691b8-8a14-42b0-aede-39126d4ab87a None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:35:58.349870 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:35:58.381137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8691b8-8a14-42b0-aede-39126d4ab87a None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:35:58.417934 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ceeb3-d8db-47e2-9738-b68f30d22078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:35:58.418592 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ceeb3-d8db-47e2-9738-b68f30d22078 None admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:35:58.418878 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ceeb3-d8db-47e2-9738-b68f30d22078 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:35:58.419179 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ceeb3-d8db-47e2-9738-b68f30d22078 None admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:35:58.419899 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ceeb3-d8db-47e2-9738-b68f30d22078 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:35:58.421659 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ceeb3-d8db-47e2-9738-b68f30d22078 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:35:58.449917 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 567/1134] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Jul 20 14:35:58 2022] POST /identity/v3/projects => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:36:00.527095 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61186f4-404b-4582-88f4-16fdcad1678e None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:00.527581 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61186f4-404b-4582-88f4-16fdcad1678e None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:00.527710 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61186f4-404b-4582-88f4-16fdcad1678e None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:00.529818 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 568/1135] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:36:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:36:00.535359 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4a61a-a9b1-4a02-8633-426585f4d8d8 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:00.535778 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4a61a-a9b1-4a02-8633-426585f4d8d8 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:00.535923 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4a61a-a9b1-4a02-8633-426585f4d8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:00.554849 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-76f4a61a-a9b1-4a02-8633-426585f4d8d8 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:36:00.556079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-76f4a61a-a9b1-4a02-8633-426585f4d8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:36:00.624760 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 568/1136] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:36:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:36:00.650461 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61186f4-404b-4582-88f4-16fdcad1678e None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:00.728118 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:00.760889 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61186f4-404b-4582-88f4-16fdcad1678e None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:00.796316 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc024665-44e2-4a66-b4c2-d51d29660670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:00.797132 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc024665-44e2-4a66-b4c2-d51d29660670 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:00.797432 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc024665-44e2-4a66-b4c2-d51d29660670 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:00.797706 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc024665-44e2-4a66-b4c2-d51d29660670 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:00.799026 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc024665-44e2-4a66-b4c2-d51d29660670 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:00.800502 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc024665-44e2-4a66-b4c2-d51d29660670 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:00.802220 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 569/1137] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:36:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:36:00.808347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f4a61a-a9b1-4a02-8633-426585f4d8d8 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:00.865028 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:00.911769 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f4a61a-a9b1-4a02-8633-426585f4d8d8 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:00.947452 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae3e8d-41db-4e21-a039-c487cd1ee925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:00.948116 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae3e8d-41db-4e21-a039-c487cd1ee925 None admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:00.948399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae3e8d-41db-4e21-a039-c487cd1ee925 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:00.948658 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae3e8d-41db-4e21-a039-c487cd1ee925 None admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:00.949385 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ae3e8d-41db-4e21-a039-c487cd1ee925 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:00.950828 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ae3e8d-41db-4e21-a039-c487cd1ee925 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:00.979178 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 569/1138] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jul 20 14:36:00 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:36:03.093438 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655bc469-862c-48e9-9915-6fea108321a1 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:03.094051 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655bc469-862c-48e9-9915-6fea108321a1 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:03.094051 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655bc469-862c-48e9-9915-6fea108321a1 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:03.095463 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 570/1139] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:36:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:36:03.103395 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730b410-69c4-4257-b753-be8e36eea2fd None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:03.103849 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730b410-69c4-4257-b753-be8e36eea2fd None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:03.104307 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730b410-69c4-4257-b753-be8e36eea2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:03.126991 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-3730b410-69c4-4257-b753-be8e36eea2fd None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:36:03.128261 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-3730b410-69c4-4257-b753-be8e36eea2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:36:03.200503 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 570/1140] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:36:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:36:03.224009 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655bc469-862c-48e9-9915-6fea108321a1 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:03.340059 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655bc469-862c-48e9-9915-6fea108321a1 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:03.375484 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a37d2-c204-46ba-955a-fc8dc48e1540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:03.376210 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a37d2-c204-46ba-955a-fc8dc48e1540 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:03.376490 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a37d2-c204-46ba-955a-fc8dc48e1540 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:03.376757 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a37d2-c204-46ba-955a-fc8dc48e1540 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:03.384155 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:03.388522 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9a37d2-c204-46ba-955a-fc8dc48e1540 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:36:03.396418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-4d9a37d2-c204-46ba-955a-fc8dc48e1540 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:36:03.396418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:36:03.396418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:36:03.396418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:36:03.396418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:36:03.396418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:36:03.396418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:36:03.396418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:36:03.396418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:36:03.396418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:36:03.396418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:36:03.396418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:36:03.396418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:36:03.396418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:36:03.396418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:36:03.396418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:36:03.397621 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:36:03.397621 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:36:03.397621 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:36:03.397621 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:36:03.397621 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:36:03.397621 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:36:03.397621 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:36:03.397621 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:36:03.397621 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:36:03.397621 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:36:03.397621 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:36:03.397621 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:36:03.397621 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:36:03.397621 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:36:03.397621 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:36:03.397621 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:36:03.398779 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:36:03.398779 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:36:03.398779 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:36:03.398779 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:36:03.398779 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:36:03.398779 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:36:03.398779 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:36:03.398779 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:36:03.398779 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:36:03.398779 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:36:03.398779 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:36:03.398779 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:36:03.398779 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 571/1141] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:36:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:36:03.404688 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730b410-69c4-4257-b753-be8e36eea2fd None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:03.450352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:03.514035 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730b410-69c4-4257-b753-be8e36eea2fd None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:03.550603 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932404a-8211-4f21-a1be-01b1c5cd6a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:03.551304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932404a-8211-4f21-a1be-01b1c5cd6a4e None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:03.551590 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932404a-8211-4f21-a1be-01b1c5cd6a4e None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:03.551868 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932404a-8211-4f21-a1be-01b1c5cd6a4e None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:03.552635 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932404a-8211-4f21-a1be-01b1c5cd6a4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:36:03.552940 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932404a-8211-4f21-a1be-01b1c5cd6a4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:03.554339 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932404a-8211-4f21-a1be-01b1c5cd6a4e None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:03.562324 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 571/1142] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:36:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:36:03.570304 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a37d2-c204-46ba-955a-fc8dc48e1540 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:03.692286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a37d2-c204-46ba-955a-fc8dc48e1540 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:03.727682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd54ecab-acd8-4c30-80a3-29dead65d7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:03.728563 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54ecab-acd8-4c30-80a3-29dead65d7e1 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:03.728850 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54ecab-acd8-4c30-80a3-29dead65d7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:03.729128 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54ecab-acd8-4c30-80a3-29dead65d7e1 None admin] PATH_INFO: `/v3/users/d05d30aa1d4142f7b7f64bf1399e72fc` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:03.742666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd54ecab-acd8-4c30-80a3-29dead65d7e1 None admin] RBAC: Authorizing `identity:get_user(user_id=d05d30aa1d4142f7b7f64bf1399e72fc)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:03.744177 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd54ecab-acd8-4c30-80a3-29dead65d7e1 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:03.745830 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 572/1143] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:36:03 2022] GET /identity/v3/users/d05d30aa1d4142f7b7f64bf1399e72fc => generated 286 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:36:03.751971 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932404a-8211-4f21-a1be-01b1c5cd6a4e None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:03.854841 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932404a-8211-4f21-a1be-01b1c5cd6a4e None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:03.856568 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:03.891330 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89743e6c-e7a2-4b43-814e-82a4c4a3d2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:03.891981 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89743e6c-e7a2-4b43-814e-82a4c4a3d2c1 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:03.892258 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89743e6c-e7a2-4b43-814e-82a4c4a3d2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:03.892539 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89743e6c-e7a2-4b43-814e-82a4c4a3d2c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:03.976898 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89743e6c-e7a2-4b43-814e-82a4c4a3d2c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:03.978283 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89743e6c-e7a2-4b43-814e-82a4c4a3d2c1 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:04.083348 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 572/1144] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:36:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 14:36:04.092106 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd54ecab-acd8-4c30-80a3-29dead65d7e1 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:04.153985 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:04.199427 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd54ecab-acd8-4c30-80a3-29dead65d7e1 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:04.235032 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c519d-e2c2-4cef-8f9a-af6c070c7692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:04.235865 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c519d-e2c2-4cef-8f9a-af6c070c7692 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:04.236145 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c519d-e2c2-4cef-8f9a-af6c070c7692 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:04.236421 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c519d-e2c2-4cef-8f9a-af6c070c7692 None admin] PATH_INFO: `/v3/projects/e60c9a1adc874699b67a804a57cb3a8c` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:04.237659 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0c519d-e2c2-4cef-8f9a-af6c070c7692 None admin] RBAC: Authorizing `identity:get_project(project_id=e60c9a1adc874699b67a804a57cb3a8c)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:04.239136 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0c519d-e2c2-4cef-8f9a-af6c070c7692 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:04.240864 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 573/1145] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:36:04 2022] GET /identity/v3/projects/e60c9a1adc874699b67a804a57cb3a8c => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:36:04.250981 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89743e6c-e7a2-4b43-814e-82a4c4a3d2c1 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:04.254416 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:04.356647 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89743e6c-e7a2-4b43-814e-82a4c4a3d2c1 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:04.393222 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b550e-3ba7-4901-8bfb-a7b6bda79b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:04.393882 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b550e-3ba7-4901-8bfb-a7b6bda79b53 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:04.394163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b550e-3ba7-4901-8bfb-a7b6bda79b53 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:04.394441 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b550e-3ba7-4901-8bfb-a7b6bda79b53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:04.395325 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b550e-3ba7-4901-8bfb-a7b6bda79b53 None admin] RBAC: Adding query filter params (role.id=49bd5abf255048a0abaa9538e309f516, scope.project.id=e60c9a1adc874699b67a804a57cb3a8c, user.id=d05d30aa1d4142f7b7f64bf1399e72fc) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:36:04.395636 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b550e-3ba7-4901-8bfb-a7b6bda79b53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:04.397124 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b550e-3ba7-4901-8bfb-a7b6bda79b53 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:04.404607 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 573/1146] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:36:04 2022] GET /identity/v3/role_assignments?user.id=d05d30aa1d4142f7b7f64bf1399e72fc&scope.project.id=e60c9a1adc874699b67a804a57cb3a8c&role.id=49bd5abf255048a0abaa9538e309f516 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:36:06.446656 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b79ab8-d47c-47eb-98ed-2506302772d9 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:06.447644 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b79ab8-d47c-47eb-98ed-2506302772d9 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:06.447644 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b79ab8-d47c-47eb-98ed-2506302772d9 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:06.449838 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 574/1147] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:36:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:36:06.455457 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73d8a10-1b44-479c-8d61-cba1e52eb89c None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:06.455876 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73d8a10-1b44-479c-8d61-cba1e52eb89c None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:06.455984 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73d8a10-1b44-479c-8d61-cba1e52eb89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:06.474649 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-d73d8a10-1b44-479c-8d61-cba1e52eb89c None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:36:06.475933 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-d73d8a10-1b44-479c-8d61-cba1e52eb89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:36:06.537159 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:06.545691 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 574/1148] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:36:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:36:06.569279 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b79ab8-d47c-47eb-98ed-2506302772d9 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:06.688492 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b79ab8-d47c-47eb-98ed-2506302772d9 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:06.713772 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:06.727495 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf786bbf-d040-4cc3-beb0-576b9872c18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:06.728307 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf786bbf-d040-4cc3-beb0-576b9872c18b None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:06.728603 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf786bbf-d040-4cc3-beb0-576b9872c18b None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:06.728894 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf786bbf-d040-4cc3-beb0-576b9872c18b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:06.739289 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf786bbf-d040-4cc3-beb0-576b9872c18b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:36:06.746941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-cf786bbf-d040-4cc3-beb0-576b9872c18b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:36:06.746941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:36:06.746941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:36:06.746941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:36:06.746941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:36:06.746941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:36:06.746941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:36:06.746941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:36:06.746941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:36:06.746941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:36:06.746941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:36:06.746941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:36:06.746941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:36:06.746941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:36:06.746941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:36:06.746941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:36:06.748072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:36:06.748072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:36:06.748072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:36:06.748072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:36:06.748072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:36:06.748072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:36:06.748072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:36:06.748072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:36:06.748072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:36:06.748072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:36:06.748072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:36:06.748072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:36:06.748072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:36:06.748072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:36:06.748072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:36:06.748072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:36:06.748945 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:36:06.748945 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:36:06.748945 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:36:06.748945 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:36:06.748945 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:36:06.748945 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:36:06.748945 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:36:06.748945 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:36:06.748945 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:36:06.748945 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:36:06.748945 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:36:06.748945 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:36:06.748945 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 575/1149] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:36:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:36:06.754679 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73d8a10-1b44-479c-8d61-cba1e52eb89c None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:06.861831 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73d8a10-1b44-479c-8d61-cba1e52eb89c None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:06.899559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b8ebd-91d3-414d-ad1a-f0adfa6c62d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:06.900222 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b8ebd-91d3-414d-ad1a-f0adfa6c62d4 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:06.900500 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b8ebd-91d3-414d-ad1a-f0adfa6c62d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:06.900775 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b8ebd-91d3-414d-ad1a-f0adfa6c62d4 None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:06.901516 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6b8ebd-91d3-414d-ad1a-f0adfa6c62d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:36:06.901822 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6b8ebd-91d3-414d-ad1a-f0adfa6c62d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:06.903282 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6b8ebd-91d3-414d-ad1a-f0adfa6c62d4 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:06.911652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 575/1150] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:36:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:36:06.917892 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf786bbf-d040-4cc3-beb0-576b9872c18b None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:07.023101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf786bbf-d040-4cc3-beb0-576b9872c18b None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:07.057823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd728d-5069-44cd-a72f-694d4b33725c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:07.058698 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd728d-5069-44cd-a72f-694d4b33725c None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:07.058981 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd728d-5069-44cd-a72f-694d4b33725c None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:07.059292 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd728d-5069-44cd-a72f-694d4b33725c None admin] PATH_INFO: `/v3/users/d05d30aa1d4142f7b7f64bf1399e72fc` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:07.060610 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbd728d-5069-44cd-a72f-694d4b33725c None admin] RBAC: Authorizing `identity:get_user(user_id=d05d30aa1d4142f7b7f64bf1399e72fc)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:07.062089 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbd728d-5069-44cd-a72f-694d4b33725c None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:07.063740 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 576/1151] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:36:06 2022] GET /identity/v3/users/d05d30aa1d4142f7b7f64bf1399e72fc => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:36:07.070532 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b8ebd-91d3-414d-ad1a-f0adfa6c62d4 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:07.136733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:07.180955 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b8ebd-91d3-414d-ad1a-f0adfa6c62d4 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:07.216048 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9244990-bced-4755-bf7a-10e2b0b02856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:07.216919 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9244990-bced-4755-bf7a-10e2b0b02856 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:07.217205 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9244990-bced-4755-bf7a-10e2b0b02856 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:07.217483 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9244990-bced-4755-bf7a-10e2b0b02856 None admin] PATH_INFO: `/v3/projects/e60c9a1adc874699b67a804a57cb3a8c` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:07.218892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9244990-bced-4755-bf7a-10e2b0b02856 None admin] RBAC: Authorizing `identity:get_project(project_id=e60c9a1adc874699b67a804a57cb3a8c)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:07.220394 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9244990-bced-4755-bf7a-10e2b0b02856 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:07.222109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 576/1152] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:36:07 2022] GET /identity/v3/projects/e60c9a1adc874699b67a804a57cb3a8c => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:36:07.232076 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd728d-5069-44cd-a72f-694d4b33725c None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:07.296640 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:07.354647 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd728d-5069-44cd-a72f-694d4b33725c None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:07.399022 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052f82d9-ed38-4c5d-b745-f0ed5ce19cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:07.399995 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f82d9-ed38-4c5d-b745-f0ed5ce19cdc None admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:07.400275 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f82d9-ed38-4c5d-b745-f0ed5ce19cdc None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:07.400552 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f82d9-ed38-4c5d-b745-f0ed5ce19cdc None admin] PATH_INFO: `/v3/projects/e60c9a1adc874699b67a804a57cb3a8c/users/d05d30aa1d4142f7b7f64bf1399e72fc/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:07.402563 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052f82d9-ed38-4c5d-b745-f0ed5ce19cdc None admin] RBAC: Authorizing `identity:create_grant(project_id=e60c9a1adc874699b67a804a57cb3a8c, user_id=d05d30aa1d4142f7b7f64bf1399e72fc, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:07.404045 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052f82d9-ed38-4c5d-b745-f0ed5ce19cdc None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:07.418902 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 577/1153] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jul 20 14:36:07 2022] PUT /identity/v3/projects/e60c9a1adc874699b67a804a57cb3a8c/users/d05d30aa1d4142f7b7f64bf1399e72fc/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:36:09.412781 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2b5c0-d7da-40ad-9c44-f262f653ac9c None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:09.413654 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2b5c0-d7da-40ad-9c44-f262f653ac9c None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:09.413654 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2b5c0-d7da-40ad-9c44-f262f653ac9c None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:09.415973 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 577/1154] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:36:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:36:09.422701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb97a05-2014-4b47-ab80-c9683773ced4 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:09.423411 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb97a05-2014-4b47-ab80-c9683773ced4 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:09.423714 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb97a05-2014-4b47-ab80-c9683773ced4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:09.448386 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-cbb97a05-2014-4b47-ab80-c9683773ced4 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:36:09.449602 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-cbb97a05-2014-4b47-ab80-c9683773ced4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:36:09.522728 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 578/1155] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:36:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:36:09.547714 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2b5c0-d7da-40ad-9c44-f262f653ac9c None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:09.664770 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2b5c0-d7da-40ad-9c44-f262f653ac9c None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:09.693040 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:09.702660 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad977bf-221d-4712-8718-6463cff831d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:09.703484 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad977bf-221d-4712-8718-6463cff831d5 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:09.703768 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad977bf-221d-4712-8718-6463cff831d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:09.704043 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad977bf-221d-4712-8718-6463cff831d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:09.715091 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad977bf-221d-4712-8718-6463cff831d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:36:09.722547 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-cad977bf-221d-4712-8718-6463cff831d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:36:09.722547 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:36:09.722547 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:36:09.722547 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:36:09.722547 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:36:09.722547 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:36:09.722547 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:36:09.722547 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:36:09.722547 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:36:09.722547 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:36:09.722547 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:36:09.722547 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:36:09.722547 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:36:09.722547 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:36:09.722547 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:36:09.722547 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:36:09.723812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:36:09.723812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:36:09.723812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:36:09.723812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:36:09.723812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:36:09.723812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:36:09.723812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:36:09.723812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:36:09.723812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:36:09.723812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:36:09.723812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:36:09.723812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:36:09.723812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:36:09.723812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:36:09.723812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:36:09.723812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:36:09.724705 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:36:09.724705 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:36:09.724705 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:36:09.724705 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:36:09.724705 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:36:09.724705 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:36:09.724705 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:36:09.724705 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:36:09.724705 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:36:09.724705 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:36:09.724705 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:36:09.724705 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:36:09.724705 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 578/1156] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:36:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:36:09.730223 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb97a05-2014-4b47-ab80-c9683773ced4 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:09.775389 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:09.838056 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb97a05-2014-4b47-ab80-c9683773ced4 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:09.874330 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5d7de-e468-406b-87de-dd3e5da2807e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:09.875014 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5d7de-e468-406b-87de-dd3e5da2807e None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:09.875331 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5d7de-e468-406b-87de-dd3e5da2807e None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:09.875608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5d7de-e468-406b-87de-dd3e5da2807e None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:09.876366 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af5d7de-e468-406b-87de-dd3e5da2807e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:36:09.876680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af5d7de-e468-406b-87de-dd3e5da2807e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:09.878082 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af5d7de-e468-406b-87de-dd3e5da2807e None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:09.886525 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 579/1157] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:36:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:36:09.895765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad977bf-221d-4712-8718-6463cff831d5 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:10.004325 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad977bf-221d-4712-8718-6463cff831d5 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:10.040379 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072650b-81f5-4b1a-97b0-13185f1146ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:10.041259 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072650b-81f5-4b1a-97b0-13185f1146ab None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:10.041540 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072650b-81f5-4b1a-97b0-13185f1146ab None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:10.041809 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072650b-81f5-4b1a-97b0-13185f1146ab None admin] PATH_INFO: `/v3/users/d05d30aa1d4142f7b7f64bf1399e72fc` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:10.043037 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f072650b-81f5-4b1a-97b0-13185f1146ab None admin] RBAC: Authorizing `identity:get_user(user_id=d05d30aa1d4142f7b7f64bf1399e72fc)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:10.044508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f072650b-81f5-4b1a-97b0-13185f1146ab None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:10.046486 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 579/1158] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:36:09 2022] GET /identity/v3/users/d05d30aa1d4142f7b7f64bf1399e72fc => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:36:10.053346 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5d7de-e468-406b-87de-dd3e5da2807e None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:10.169845 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5d7de-e468-406b-87de-dd3e5da2807e None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:10.172701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:10.206379 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b81cc-d7ee-417e-b2c0-c4a2d34ae850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:10.207187 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b81cc-d7ee-417e-b2c0-c4a2d34ae850 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:10.207397 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b81cc-d7ee-417e-b2c0-c4a2d34ae850 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:10.207781 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b81cc-d7ee-417e-b2c0-c4a2d34ae850 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:10.277529 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b81cc-d7ee-417e-b2c0-c4a2d34ae850 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:10.278931 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b81cc-d7ee-417e-b2c0-c4a2d34ae850 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:10.385658 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 580/1159] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:36:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 14:36:10.392212 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072650b-81f5-4b1a-97b0-13185f1146ab None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:10.458863 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:10.515589 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072650b-81f5-4b1a-97b0-13185f1146ab None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:10.555354 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65bcfb5-9722-4ba1-b36e-3c2d2afb1b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:10.556229 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bcfb5-9722-4ba1-b36e-3c2d2afb1b51 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:10.556513 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bcfb5-9722-4ba1-b36e-3c2d2afb1b51 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:10.556787 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bcfb5-9722-4ba1-b36e-3c2d2afb1b51 None admin] PATH_INFO: `/v3/projects/e60c9a1adc874699b67a804a57cb3a8c` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:10.558110 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65bcfb5-9722-4ba1-b36e-3c2d2afb1b51 None admin] RBAC: Authorizing `identity:get_project(project_id=e60c9a1adc874699b67a804a57cb3a8c)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:10.559632 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65bcfb5-9722-4ba1-b36e-3c2d2afb1b51 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:10.561358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 580/1160] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:36:10 2022] GET /identity/v3/projects/e60c9a1adc874699b67a804a57cb3a8c => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:36:10.567939 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b81cc-d7ee-417e-b2c0-c4a2d34ae850 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:10.571260 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:10.672876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b81cc-d7ee-417e-b2c0-c4a2d34ae850 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:10.714272 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c5310-c2f3-4d31-a339-a05ef9ffadc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:10.714939 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c5310-c2f3-4d31-a339-a05ef9ffadc5 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:10.715263 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c5310-c2f3-4d31-a339-a05ef9ffadc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:10.715537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c5310-c2f3-4d31-a339-a05ef9ffadc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:10.716407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c5310-c2f3-4d31-a339-a05ef9ffadc5 None admin] RBAC: Adding query filter params (role.id=49bd5abf255048a0abaa9538e309f516, scope.project.id=e60c9a1adc874699b67a804a57cb3a8c, user.id=d05d30aa1d4142f7b7f64bf1399e72fc) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:36:10.716726 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c5310-c2f3-4d31-a339-a05ef9ffadc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:10.718172 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c5310-c2f3-4d31-a339-a05ef9ffadc5 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:10.724203 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 581/1161] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:36:10 2022] GET /identity/v3/role_assignments?user.id=d05d30aa1d4142f7b7f64bf1399e72fc&scope.project.id=e60c9a1adc874699b67a804a57cb3a8c&role.id=49bd5abf255048a0abaa9538e309f516 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:36:12.785438 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcc77a-61df-451f-9bac-bb93b49edfd0 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:12.786249 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcc77a-61df-451f-9bac-bb93b49edfd0 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:12.786249 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcc77a-61df-451f-9bac-bb93b49edfd0 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:12.788486 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 581/1162] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:36:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:36:12.794017 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b682390-d808-438a-a61f-f6f2ae172559 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:12.794452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b682390-d808-438a-a61f-f6f2ae172559 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:12.794550 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b682390-d808-438a-a61f-f6f2ae172559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:12.813764 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-3b682390-d808-438a-a61f-f6f2ae172559 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:36:12.814956 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-3b682390-d808-438a-a61f-f6f2ae172559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:36:12.884317 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:12.890959 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 582/1163] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:36:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:36:12.913653 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bcc77a-61df-451f-9bac-bb93b49edfd0 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:13.040797 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bcc77a-61df-451f-9bac-bb93b49edfd0 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:13.066233 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:13.080709 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b364d0-ebd1-4b12-808c-8bba86af1752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:13.081362 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b364d0-ebd1-4b12-808c-8bba86af1752 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:13.081646 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b364d0-ebd1-4b12-808c-8bba86af1752 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:13.081926 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b364d0-ebd1-4b12-808c-8bba86af1752 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:13.152853 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b364d0-ebd1-4b12-808c-8bba86af1752 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:13.154304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b364d0-ebd1-4b12-808c-8bba86af1752 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:13.266140 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 582/1164] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:36:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 14:36:13.272524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b682390-d808-438a-a61f-f6f2ae172559 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:13.377868 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b682390-d808-438a-a61f-f6f2ae172559 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:13.416447 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df03fa-69f7-4f6c-8555-c3fb1f0d41a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:13.417255 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df03fa-69f7-4f6c-8555-c3fb1f0d41a1 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:13.417537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df03fa-69f7-4f6c-8555-c3fb1f0d41a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:13.417811 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df03fa-69f7-4f6c-8555-c3fb1f0d41a1 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:13.425773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df03fa-69f7-4f6c-8555-c3fb1f0d41a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:13.427324 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df03fa-69f7-4f6c-8555-c3fb1f0d41a1 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:13.442862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-05df03fa-69f7-4f6c-8555-c3fb1f0d41a1 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 20 14:36:13.442862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:36:13.442862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 20 14:36:13.442862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 20 14:36:13.442862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:36:13.442862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:36:13.442862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:36:13.442862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:36:13.442862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 20 14:36:13.442862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:36:13.442862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 14:36:13.442862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:36:13.442862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:36:13.442862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:36:13.442862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:36:13.442862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:36:13.442862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:36:13.444157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:36:13.444157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:36:13.444157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:36:13.444157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:36:13.444157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:36:13.444157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:36:13.444157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:36:13.444157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:36:13.444157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:36:13.444157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:36:13.444157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:36:13.444157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:36:13.444157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:36:13.444157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:36:13.444157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:36:13.444157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:36:13.445242 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:36:13.445242 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:36:13.445242 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:36:13.445242 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:36:13.445242 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:36:13.445242 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:36:13.445242 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:36:13.445242 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:36:13.445242 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:36:13.445242 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:36:13.445242 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 20 14:36:13.445242 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:36:13.445242 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 20 14:36:13.445242 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:36:13.445242 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 583/1165] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Jul 20 14:36:13 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:36:13.451089 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b364d0-ebd1-4b12-808c-8bba86af1752 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:13.484517 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:13.567646 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b364d0-ebd1-4b12-808c-8bba86af1752 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:13.603375 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6756484-4ef0-4017-a66f-3da0b65b0b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:13.604369 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6756484-4ef0-4017-a66f-3da0b65b0b85 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:13.604653 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6756484-4ef0-4017-a66f-3da0b65b0b85 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:13.604930 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6756484-4ef0-4017-a66f-3da0b65b0b85 None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:13.605659 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6756484-4ef0-4017-a66f-3da0b65b0b85 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:36:13.605967 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6756484-4ef0-4017-a66f-3da0b65b0b85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:13.607377 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6756484-4ef0-4017-a66f-3da0b65b0b85 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:13.615020 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 583/1166] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Jul 20 14:36:13 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:36:15.624411 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f17d87-ebc1-4819-adf2-0729ea854e67 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:15.625283 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f17d87-ebc1-4819-adf2-0729ea854e67 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:15.625283 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f17d87-ebc1-4819-adf2-0729ea854e67 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:15.627621 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 584/1167] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:36:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:36:15.635613 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8640f1f-9cb8-4d7e-8583-30f53b0ea642 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:15.636051 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8640f1f-9cb8-4d7e-8583-30f53b0ea642 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:15.636140 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8640f1f-9cb8-4d7e-8583-30f53b0ea642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:15.655617 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-b8640f1f-9cb8-4d7e-8583-30f53b0ea642 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:36:15.656807 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-b8640f1f-9cb8-4d7e-8583-30f53b0ea642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:36:15.728466 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 584/1168] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:36:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 14:36:15.752245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f17d87-ebc1-4819-adf2-0729ea854e67 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:15.826361 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:15.875209 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f17d87-ebc1-4819-adf2-0729ea854e67 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:15.913402 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501feb68-ba83-4232-969e-24641d869f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:15.914066 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501feb68-ba83-4232-969e-24641d869f26 None admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:15.914349 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501feb68-ba83-4232-969e-24641d869f26 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:15.914630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501feb68-ba83-4232-969e-24641d869f26 None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:15.915426 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501feb68-ba83-4232-969e-24641d869f26 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:15.916801 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501feb68-ba83-4232-969e-24641d869f26 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:15.941816 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 585/1169] 10.222.0.15 () {60 vars in 1138 bytes} [Wed Jul 20 14:36:15 2022] POST /identity/v3/domains => generated 267 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:36:17.998919 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ee7da-a011-476b-85e3-1dc4ab8202f9 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:17.999644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ee7da-a011-476b-85e3-1dc4ab8202f9 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:17.999644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ee7da-a011-476b-85e3-1dc4ab8202f9 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:18.000909 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 585/1170] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:36:18.007459 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f3ed3-a257-4eb0-a81f-4d1e01984042 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:18.008212 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f3ed3-a257-4eb0-a81f-4d1e01984042 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:18.008518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f3ed3-a257-4eb0-a81f-4d1e01984042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:18.032033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-e76f3ed3-a257-4eb0-a81f-4d1e01984042 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:36:18.033325 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e76f3ed3-a257-4eb0-a81f-4d1e01984042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:36:18.105555 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 586/1171] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:36:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:36:18.128992 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ee7da-a011-476b-85e3-1dc4ab8202f9 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:18.150403 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:18.257778 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ee7da-a011-476b-85e3-1dc4ab8202f9 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:18.293020 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b9c92-bc55-480b-bb03-95505388a4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:18.293885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b9c92-bc55-480b-bb03-95505388a4f5 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:18.294163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b9c92-bc55-480b-bb03-95505388a4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:18.294434 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b9c92-bc55-480b-bb03-95505388a4f5 None admin] PATH_INFO: `/v3/domains/c49b6d9e45b846abbc451e008ba9cda6` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:18.305944 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b9c92-bc55-480b-bb03-95505388a4f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=c49b6d9e45b846abbc451e008ba9cda6)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:18.307461 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b9c92-bc55-480b-bb03-95505388a4f5 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:18.309124 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 586/1172] 10.222.0.15 () {56 vars in 1183 bytes} [Wed Jul 20 14:36:18 2022] GET /identity/v3/domains/c49b6d9e45b846abbc451e008ba9cda6 => generated 267 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:36:18.318064 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f3ed3-a257-4eb0-a81f-4d1e01984042 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:18.369863 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:18.428031 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f3ed3-a257-4eb0-a81f-4d1e01984042 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:18.476035 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a3810-7fe6-4c80-879d-fcdb3e4f5c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:18.476716 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a3810-7fe6-4c80-879d-fcdb3e4f5c3a None admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:18.476998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a3810-7fe6-4c80-879d-fcdb3e4f5c3a None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:18.477282 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a3810-7fe6-4c80-879d-fcdb3e4f5c3a None admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:18.478014 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a3810-7fe6-4c80-879d-fcdb3e4f5c3a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:18.479501 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a3810-7fe6-4c80-879d-fcdb3e4f5c3a None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:18.509591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 587/1173] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Jul 20 14:36:18 2022] POST /identity/v3/projects => generated 361 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 14:36:20.563310 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c2a11-1cbd-4994-8eb1-aecefd4d0951 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:20.564230 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c2a11-1cbd-4994-8eb1-aecefd4d0951 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:20.564230 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c2a11-1cbd-4994-8eb1-aecefd4d0951 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:20.566498 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 587/1174] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:36:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:36:20.572545 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d80a2d-ccf3-41e3-b59e-a925a5094a0c None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:20.573101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d80a2d-ccf3-41e3-b59e-a925a5094a0c None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:20.573101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d80a2d-ccf3-41e3-b59e-a925a5094a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:20.592225 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-e8d80a2d-ccf3-41e3-b59e-a925a5094a0c None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:36:20.593420 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e8d80a2d-ccf3-41e3-b59e-a925a5094a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:36:20.671154 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 588/1175] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:36:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:36:20.696761 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c2a11-1cbd-4994-8eb1-aecefd4d0951 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:20.792089 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:20.819214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c2a11-1cbd-4994-8eb1-aecefd4d0951 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:20.853364 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a378e-c710-41d4-8400-bbe2cf1befc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:20.854221 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a378e-c710-41d4-8400-bbe2cf1befc2 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:20.854502 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a378e-c710-41d4-8400-bbe2cf1befc2 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:20.854750 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a378e-c710-41d4-8400-bbe2cf1befc2 None admin] PATH_INFO: `/v3/domains/c49b6d9e45b846abbc451e008ba9cda6` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:20.856020 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a378e-c710-41d4-8400-bbe2cf1befc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=c49b6d9e45b846abbc451e008ba9cda6)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:20.857467 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a378e-c710-41d4-8400-bbe2cf1befc2 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:20.859103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 588/1176] 10.222.0.15 () {56 vars in 1183 bytes} [Wed Jul 20 14:36:20 2022] GET /identity/v3/domains/c49b6d9e45b846abbc451e008ba9cda6 => generated 267 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:36:20.865382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d80a2d-ccf3-41e3-b59e-a925a5094a0c None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:20.897077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:20.983273 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d80a2d-ccf3-41e3-b59e-a925a5094a0c None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:21.035311 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0545250-7019-40b2-946f-ef7a4da3424e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:21.035989 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0545250-7019-40b2-946f-ef7a4da3424e None admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:21.036270 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0545250-7019-40b2-946f-ef7a4da3424e None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:21.036542 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0545250-7019-40b2-946f-ef7a4da3424e None admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:21.037259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0545250-7019-40b2-946f-ef7a4da3424e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:21.038724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0545250-7019-40b2-946f-ef7a4da3424e None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:21.065049 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 589/1177] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jul 20 14:36:20 2022] POST /identity/v3/users => generated 311 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:36:23.099161 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d286cf-df40-4480-bcc0-8a1dcf2e6100 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:23.100018 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d286cf-df40-4480-bcc0-8a1dcf2e6100 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:23.100152 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d286cf-df40-4480-bcc0-8a1dcf2e6100 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:23.102649 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 589/1178] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:36:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:36:23.109454 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f9c3c-5b48-4e40-bb14-6acece10cfec None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:23.110224 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f9c3c-5b48-4e40-bb14-6acece10cfec None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:23.110376 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f9c3c-5b48-4e40-bb14-6acece10cfec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:23.135952 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-fc7f9c3c-5b48-4e40-bb14-6acece10cfec None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:36:23.137171 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-fc7f9c3c-5b48-4e40-bb14-6acece10cfec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:36:23.208531 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 590/1179] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:36:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:36:23.244810 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d286cf-df40-4480-bcc0-8a1dcf2e6100 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:23.356859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d286cf-df40-4480-bcc0-8a1dcf2e6100 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:23.393889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b05c7a5-516c-4130-bcac-d3af5244ebe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:23.394696 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05c7a5-516c-4130-bcac-d3af5244ebe1 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:23.394979 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05c7a5-516c-4130-bcac-d3af5244ebe1 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:23.395280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05c7a5-516c-4130-bcac-d3af5244ebe1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:23.406229 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b05c7a5-516c-4130-bcac-d3af5244ebe1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:36:23.414039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-5b05c7a5-516c-4130-bcac-d3af5244ebe1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:36:23.414039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:36:23.414039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:36:23.414039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:36:23.414039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:36:23.414039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:36:23.414039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:36:23.414039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:36:23.414039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:36:23.414039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:36:23.414039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:36:23.414039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:36:23.414039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:36:23.414039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:36:23.414039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:36:23.414039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:36:23.415301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:36:23.415301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:36:23.415301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:36:23.415301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:36:23.415301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:36:23.415301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:36:23.415301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:36:23.415301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:36:23.415301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:36:23.415301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:36:23.415301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:36:23.415301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:36:23.415301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:36:23.415301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:36:23.415301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:36:23.415301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:36:23.416328 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:36:23.416328 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:36:23.416328 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:36:23.416328 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:36:23.416328 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:36:23.416328 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:36:23.416328 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:36:23.416328 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:36:23.416328 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:36:23.416328 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:36:23.416328 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:36:23.416328 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:36:23.416328 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 590/1180] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:36:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:36:23.421765 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f9c3c-5b48-4e40-bb14-6acece10cfec None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:23.441064 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:23.530803 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f9c3c-5b48-4e40-bb14-6acece10cfec None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:23.568113 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df6d47-1cf6-478f-a0f9-9b140dde7f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:23.568887 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df6d47-1cf6-478f-a0f9-9b140dde7f45 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:23.569469 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df6d47-1cf6-478f-a0f9-9b140dde7f45 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:23.569469 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df6d47-1cf6-478f-a0f9-9b140dde7f45 None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:23.570202 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46df6d47-1cf6-478f-a0f9-9b140dde7f45 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:36:23.570518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46df6d47-1cf6-478f-a0f9-9b140dde7f45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:23.571934 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46df6d47-1cf6-478f-a0f9-9b140dde7f45 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:23.580222 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 591/1181] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:36:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:36:23.586490 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b05c7a5-516c-4130-bcac-d3af5244ebe1 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:23.590517 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:23.709380 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b05c7a5-516c-4130-bcac-d3af5244ebe1 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:23.744987 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28444cad-9bcb-4476-bb15-7dffc7bb0678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:23.745876 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28444cad-9bcb-4476-bb15-7dffc7bb0678 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:23.746164 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28444cad-9bcb-4476-bb15-7dffc7bb0678 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:23.746444 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28444cad-9bcb-4476-bb15-7dffc7bb0678 None admin] PATH_INFO: `/v3/users/269466e006ea4c018264fdd660cdc387` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:23.760525 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28444cad-9bcb-4476-bb15-7dffc7bb0678 None admin] RBAC: Authorizing `identity:get_user(user_id=269466e006ea4c018264fdd660cdc387)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:23.762001 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28444cad-9bcb-4476-bb15-7dffc7bb0678 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:23.763711 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 591/1182] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:36:23 2022] GET /identity/v3/users/269466e006ea4c018264fdd660cdc387 => generated 311 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:36:23.769893 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df6d47-1cf6-478f-a0f9-9b140dde7f45 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:23.858903 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:23.884658 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df6d47-1cf6-478f-a0f9-9b140dde7f45 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:23.919939 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38924210-a3a1-4449-a211-09a86b31553d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:23.920974 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38924210-a3a1-4449-a211-09a86b31553d None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:23.921271 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38924210-a3a1-4449-a211-09a86b31553d None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:23.921544 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38924210-a3a1-4449-a211-09a86b31553d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:23.995335 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38924210-a3a1-4449-a211-09a86b31553d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:23.996731 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38924210-a3a1-4449-a211-09a86b31553d None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:24.080336 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:24.105744 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 592/1183] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:36:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 14:36:24.112585 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28444cad-9bcb-4476-bb15-7dffc7bb0678 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:24.198935 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:24.254316 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28444cad-9bcb-4476-bb15-7dffc7bb0678 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:24.293108 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d4be02-5c55-4a19-b237-b2242da5ae86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:24.294012 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4be02-5c55-4a19-b237-b2242da5ae86 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:24.294295 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4be02-5c55-4a19-b237-b2242da5ae86 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:24.294567 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4be02-5c55-4a19-b237-b2242da5ae86 None admin] PATH_INFO: `/v3/projects/571623000f864440b01f62b18f48f060` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:24.296254 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d4be02-5c55-4a19-b237-b2242da5ae86 None admin] RBAC: Authorizing `identity:get_project(project_id=571623000f864440b01f62b18f48f060)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:24.297766 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d4be02-5c55-4a19-b237-b2242da5ae86 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:24.300780 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 592/1184] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:36:24 2022] GET /identity/v3/projects/571623000f864440b01f62b18f48f060 => generated 361 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:36:24.310482 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38924210-a3a1-4449-a211-09a86b31553d None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:24.428984 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38924210-a3a1-4449-a211-09a86b31553d None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:24.471037 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6df9bf-4e57-4e95-b18c-06a25c28bdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:24.471722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6df9bf-4e57-4e95-b18c-06a25c28bdb9 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:24.472009 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6df9bf-4e57-4e95-b18c-06a25c28bdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:24.472287 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6df9bf-4e57-4e95-b18c-06a25c28bdb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:24.473129 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6df9bf-4e57-4e95-b18c-06a25c28bdb9 None admin] RBAC: Adding query filter params (role.id=49bd5abf255048a0abaa9538e309f516, scope.project.id=571623000f864440b01f62b18f48f060, user.id=269466e006ea4c018264fdd660cdc387) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:36:24.473442 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6df9bf-4e57-4e95-b18c-06a25c28bdb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:24.474974 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6df9bf-4e57-4e95-b18c-06a25c28bdb9 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:24.481019 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 593/1185] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:36:24 2022] GET /identity/v3/role_assignments?user.id=269466e006ea4c018264fdd660cdc387&scope.project.id=571623000f864440b01f62b18f48f060&role.id=49bd5abf255048a0abaa9538e309f516 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:36:26.473217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a1b10-d492-49c5-817f-7b1662b2a711 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:26.473815 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a1b10-d492-49c5-817f-7b1662b2a711 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:26.473815 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a1b10-d492-49c5-817f-7b1662b2a711 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:26.478716 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 593/1186] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:36:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 14:36:26.482867 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdb9c9-83d2-4887-a7ae-bc960c2ef931 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:26.483368 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdb9c9-83d2-4887-a7ae-bc960c2ef931 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:26.483519 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdb9c9-83d2-4887-a7ae-bc960c2ef931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:26.504359 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-96cdb9c9-83d2-4887-a7ae-bc960c2ef931 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:36:26.505561 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-96cdb9c9-83d2-4887-a7ae-bc960c2ef931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:36:26.553106 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:26.587191 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 594/1187] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:36:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:36:26.610412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a1b10-d492-49c5-817f-7b1662b2a711 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:26.725666 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a1b10-d492-49c5-817f-7b1662b2a711 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:26.753836 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:26.761145 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc5d62-fed6-4d95-ad17-affb8886d64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:26.761944 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc5d62-fed6-4d95-ad17-affb8886d64e None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:26.762227 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc5d62-fed6-4d95-ad17-affb8886d64e None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:26.762502 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc5d62-fed6-4d95-ad17-affb8886d64e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:26.773808 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc5d62-fed6-4d95-ad17-affb8886d64e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:36:26.781381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-f5dc5d62-fed6-4d95-ad17-affb8886d64e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:36:26.781381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:36:26.781381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:36:26.781381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:36:26.781381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:36:26.781381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:36:26.781381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:36:26.781381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:36:26.781381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:36:26.781381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:36:26.781381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:36:26.781381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:36:26.781381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:36:26.781381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:36:26.781381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:36:26.781381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:36:26.782524 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:36:26.782524 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:36:26.782524 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:36:26.782524 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:36:26.782524 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:36:26.782524 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:36:26.782524 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:36:26.782524 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:36:26.782524 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:36:26.782524 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:36:26.782524 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:36:26.782524 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:36:26.782524 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:36:26.782524 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:36:26.782524 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:36:26.782524 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:36:26.784116 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:36:26.784116 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:36:26.784116 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:36:26.784116 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:36:26.784116 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:36:26.784116 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:36:26.784116 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:36:26.784116 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:36:26.784116 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:36:26.784116 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:36:26.784116 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:36:26.784116 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:36:26.784116 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 594/1188] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:36:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:36:26.790860 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cdb9c9-83d2-4887-a7ae-bc960c2ef931 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:26.905888 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cdb9c9-83d2-4887-a7ae-bc960c2ef931 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:26.944789 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751197e9-a984-4d2a-82bc-4a9bdd1e1b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:26.945476 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751197e9-a984-4d2a-82bc-4a9bdd1e1b81 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:26.945762 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751197e9-a984-4d2a-82bc-4a9bdd1e1b81 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:26.946042 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751197e9-a984-4d2a-82bc-4a9bdd1e1b81 None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:26.946800 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751197e9-a984-4d2a-82bc-4a9bdd1e1b81 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:36:26.947116 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751197e9-a984-4d2a-82bc-4a9bdd1e1b81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:26.948537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751197e9-a984-4d2a-82bc-4a9bdd1e1b81 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:26.957132 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 595/1189] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:36:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:36:26.963839 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc5d62-fed6-4d95-ad17-affb8886d64e None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:27.072031 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc5d62-fed6-4d95-ad17-affb8886d64e None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:27.119694 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7be500-f055-4ed3-83c9-8e348364c7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:27.120579 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7be500-f055-4ed3-83c9-8e348364c7b7 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:27.120864 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7be500-f055-4ed3-83c9-8e348364c7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:27.121141 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7be500-f055-4ed3-83c9-8e348364c7b7 None admin] PATH_INFO: `/v3/users/269466e006ea4c018264fdd660cdc387` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:27.122507 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7be500-f055-4ed3-83c9-8e348364c7b7 None admin] RBAC: Authorizing `identity:get_user(user_id=269466e006ea4c018264fdd660cdc387)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:27.124010 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7be500-f055-4ed3-83c9-8e348364c7b7 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:27.125660 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 595/1190] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:36:26 2022] GET /identity/v3/users/269466e006ea4c018264fdd660cdc387 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:36:27.134468 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751197e9-a984-4d2a-82bc-4a9bdd1e1b81 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:27.171639 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:27.244033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751197e9-a984-4d2a-82bc-4a9bdd1e1b81 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:27.279386 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576ea40b-bd02-4fe9-a2e9-83775692bc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:27.280271 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ea40b-bd02-4fe9-a2e9-83775692bc0a None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:27.280552 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ea40b-bd02-4fe9-a2e9-83775692bc0a None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:27.280836 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ea40b-bd02-4fe9-a2e9-83775692bc0a None admin] PATH_INFO: `/v3/projects/571623000f864440b01f62b18f48f060` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:27.282047 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576ea40b-bd02-4fe9-a2e9-83775692bc0a None admin] RBAC: Authorizing `identity:get_project(project_id=571623000f864440b01f62b18f48f060)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:27.283546 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576ea40b-bd02-4fe9-a2e9-83775692bc0a None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:27.285232 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 596/1191] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:36:27 2022] GET /identity/v3/projects/571623000f864440b01f62b18f48f060 => generated 361 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:36:27.292070 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7be500-f055-4ed3-83c9-8e348364c7b7 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:27.356923 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:27.403388 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7be500-f055-4ed3-83c9-8e348364c7b7 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:27.440230 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9806ee5-0b62-4dc5-883b-04e144204c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:27.441131 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9806ee5-0b62-4dc5-883b-04e144204c55 None admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:27.441426 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9806ee5-0b62-4dc5-883b-04e144204c55 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:27.441706 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9806ee5-0b62-4dc5-883b-04e144204c55 None admin] PATH_INFO: `/v3/projects/571623000f864440b01f62b18f48f060/users/269466e006ea4c018264fdd660cdc387/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:27.443938 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9806ee5-0b62-4dc5-883b-04e144204c55 None admin] RBAC: Authorizing `identity:create_grant(project_id=571623000f864440b01f62b18f48f060, user_id=269466e006ea4c018264fdd660cdc387, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:27.445385 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9806ee5-0b62-4dc5-883b-04e144204c55 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:27.461429 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 596/1192] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jul 20 14:36:27 2022] PUT /identity/v3/projects/571623000f864440b01f62b18f48f060/users/269466e006ea4c018264fdd660cdc387/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:36:29.471971 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf21426f-b364-458f-b0d9-24fe0eb63b15 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:29.472570 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf21426f-b364-458f-b0d9-24fe0eb63b15 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:29.472570 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf21426f-b364-458f-b0d9-24fe0eb63b15 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:29.473984 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 597/1193] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:36:29.479648 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e905a-c249-402e-bf44-ec6e11daea9a None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:29.480208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e905a-c249-402e-bf44-ec6e11daea9a None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:29.480208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e905a-c249-402e-bf44-ec6e11daea9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:29.501862 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-967e905a-c249-402e-bf44-ec6e11daea9a None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:36:29.503099 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-967e905a-c249-402e-bf44-ec6e11daea9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:36:29.582141 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 597/1194] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:36:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:36:29.605813 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf21426f-b364-458f-b0d9-24fe0eb63b15 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:29.719778 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf21426f-b364-458f-b0d9-24fe0eb63b15 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:29.721362 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:29.758382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f13b9-2beb-4a2d-b153-13fd17a7f30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:29.759099 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f13b9-2beb-4a2d-b153-13fd17a7f30c None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:29.759421 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f13b9-2beb-4a2d-b153-13fd17a7f30c None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:29.759696 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f13b9-2beb-4a2d-b153-13fd17a7f30c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:29.772611 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f13b9-2beb-4a2d-b153-13fd17a7f30c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:36:29.780681 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-d35f13b9-2beb-4a2d-b153-13fd17a7f30c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:36:29.780681 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:36:29.780681 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:36:29.780681 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:36:29.780681 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:36:29.780681 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:36:29.780681 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:36:29.780681 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:36:29.780681 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:36:29.780681 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:36:29.780681 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:36:29.780681 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:36:29.780681 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:36:29.780681 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:36:29.780681 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:36:29.780681 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:36:29.781773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:36:29.781773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:36:29.781773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:36:29.781773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:36:29.781773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:36:29.781773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:36:29.781773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:36:29.781773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:36:29.781773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:36:29.781773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:36:29.781773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:36:29.781773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:36:29.781773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:36:29.781773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:36:29.781773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:36:29.781773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:36:29.782810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:36:29.782810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:36:29.782810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:36:29.782810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:36:29.782810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:36:29.782810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:36:29.782810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:36:29.782810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:36:29.782810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:36:29.782810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:36:29.782810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:36:29.782810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:36:29.782810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 598/1195] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:36:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:36:29.788282 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967e905a-c249-402e-bf44-ec6e11daea9a None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:29.853553 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:29.911702 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967e905a-c249-402e-bf44-ec6e11daea9a None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:29.950240 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b4547-0495-4b18-9a1e-3b4a1b4c7f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:29.950905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b4547-0495-4b18-9a1e-3b4a1b4c7f8e None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:29.951220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b4547-0495-4b18-9a1e-3b4a1b4c7f8e None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:29.951488 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b4547-0495-4b18-9a1e-3b4a1b4c7f8e None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:29.952242 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b4547-0495-4b18-9a1e-3b4a1b4c7f8e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:36:29.952548 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b4547-0495-4b18-9a1e-3b4a1b4c7f8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:29.953925 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b4547-0495-4b18-9a1e-3b4a1b4c7f8e None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:29.962612 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 598/1196] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:36:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:36:29.971959 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f13b9-2beb-4a2d-b153-13fd17a7f30c None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:30.072830 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f13b9-2beb-4a2d-b153-13fd17a7f30c None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:30.108106 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb66e0bf-95df-4121-920f-f1e04033cba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:30.108985 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66e0bf-95df-4121-920f-f1e04033cba6 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:30.109266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66e0bf-95df-4121-920f-f1e04033cba6 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:30.109543 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66e0bf-95df-4121-920f-f1e04033cba6 None admin] PATH_INFO: `/v3/users/269466e006ea4c018264fdd660cdc387` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:30.110888 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb66e0bf-95df-4121-920f-f1e04033cba6 None admin] RBAC: Authorizing `identity:get_user(user_id=269466e006ea4c018264fdd660cdc387)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:30.112348 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb66e0bf-95df-4121-920f-f1e04033cba6 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:30.113987 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 599/1197] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jul 20 14:36:29 2022] GET /identity/v3/users/269466e006ea4c018264fdd660cdc387 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:36:30.120091 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b4547-0495-4b18-9a1e-3b4a1b4c7f8e None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:30.241487 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b4547-0495-4b18-9a1e-3b4a1b4c7f8e None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:30.252986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:30.284448 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07a1ef-6dd2-418e-b592-7caf4d22d132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:30.285467 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07a1ef-6dd2-418e-b592-7caf4d22d132 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:30.285828 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07a1ef-6dd2-418e-b592-7caf4d22d132 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:30.286212 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07a1ef-6dd2-418e-b592-7caf4d22d132 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:30.358459 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a07a1ef-6dd2-418e-b592-7caf4d22d132 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:30.359873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a07a1ef-6dd2-418e-b592-7caf4d22d132 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:30.465457 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 599/1198] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:36:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 14:36:30.471663 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb66e0bf-95df-4121-920f-f1e04033cba6 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:30.505642 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:30.581905 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb66e0bf-95df-4121-920f-f1e04033cba6 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:30.620360 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc463e03-0886-4f35-b847-12c2af3cb069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:30.621166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc463e03-0886-4f35-b847-12c2af3cb069 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:30.621448 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc463e03-0886-4f35-b847-12c2af3cb069 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:30.621718 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc463e03-0886-4f35-b847-12c2af3cb069 None admin] PATH_INFO: `/v3/projects/571623000f864440b01f62b18f48f060` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:30.623025 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc463e03-0886-4f35-b847-12c2af3cb069 None admin] RBAC: Authorizing `identity:get_project(project_id=571623000f864440b01f62b18f48f060)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:30.624517 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc463e03-0886-4f35-b847-12c2af3cb069 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:30.626230 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 600/1199] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jul 20 14:36:30 2022] GET /identity/v3/projects/571623000f864440b01f62b18f48f060 => generated 361 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:36:30.633238 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07a1ef-6dd2-418e-b592-7caf4d22d132 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:36:30.650684 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:30.746139 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07a1ef-6dd2-418e-b592-7caf4d22d132 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:36:30.782292 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2790f42-7d9a-460c-8b31-a4183ca5a5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:36:30.782943 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2790f42-7d9a-460c-8b31-a4183ca5a5d7 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:36:30.783257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2790f42-7d9a-460c-8b31-a4183ca5a5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:36:30.783553 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2790f42-7d9a-460c-8b31-a4183ca5a5d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:36:30.784363 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2790f42-7d9a-460c-8b31-a4183ca5a5d7 None admin] RBAC: Adding query filter params (role.id=49bd5abf255048a0abaa9538e309f516, scope.project.id=571623000f864440b01f62b18f48f060, user.id=269466e006ea4c018264fdd660cdc387) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:36:30.784672 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2790f42-7d9a-460c-8b31-a4183ca5a5d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:36:30.786117 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2790f42-7d9a-460c-8b31-a4183ca5a5d7 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:36:30.792368 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 600/1200] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:36:30 2022] GET /identity/v3/role_assignments?user.id=269466e006ea4c018264fdd660cdc387&scope.project.id=571623000f864440b01f62b18f48f060&role.id=49bd5abf255048a0abaa9538e309f516 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:36:40.615002 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=51 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:36:40.789481 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=63 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:26.381638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e604fd1-0444-44d9-a216-66dc265ec384 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:26.382286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e604fd1-0444-44d9-a216-66dc265ec384 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:26.382286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e604fd1-0444-44d9-a216-66dc265ec384 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:26.383848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 601/1201] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:37:26.389203 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62907a55-73e9-46d7-aa1f-1675824c165b None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:26.389760 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62907a55-73e9-46d7-aa1f-1675824c165b None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:26.389760 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62907a55-73e9-46d7-aa1f-1675824c165b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:26.416754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-62907a55-73e9-46d7-aa1f-1675824c165b None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:37:26.418854 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-62907a55-73e9-46d7-aa1f-1675824c165b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:37:26.523391 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 601/1202] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:37:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:37:26.548372 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e604fd1-0444-44d9-a216-66dc265ec384 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:26.672511 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e604fd1-0444-44d9-a216-66dc265ec384 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:26.717389 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d283c86a-b5e0-4045-8f28-1ea80af31d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:26.718202 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283c86a-b5e0-4045-8f28-1ea80af31d5c None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:26.718517 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283c86a-b5e0-4045-8f28-1ea80af31d5c None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:26.718807 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283c86a-b5e0-4045-8f28-1ea80af31d5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:26.733251 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d283c86a-b5e0-4045-8f28-1ea80af31d5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:26.734824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d283c86a-b5e0-4045-8f28-1ea80af31d5c None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:26.758455 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-d283c86a-b5e0-4045-8f28-1ea80af31d5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:37:26.758455 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:26.758455 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:37:26.758455 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:37:26.758455 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:37:26.758455 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:37:26.758455 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:37:26.758455 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:37:26.758455 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:37:26.758455 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:37:26.758455 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:37:26.758455 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:37:26.758455 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:37:26.758455 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:37:26.758455 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:37:26.759562 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:37:26.759562 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:37:26.759562 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:37:26.759562 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:37:26.759562 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:37:26.759562 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:37:26.759562 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:37:26.759562 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:37:26.759562 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:37:26.759562 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:37:26.759562 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:37:26.759562 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:37:26.759562 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:37:26.759562 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:37:26.759562 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:37:26.759562 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:37:26.759562 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:37:26.760617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:37:26.760617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:37:26.760617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:37:26.760617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:37:26.763903 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 602/1203] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:37:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:37:26.771629 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62907a55-73e9-46d7-aa1f-1675824c165b None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:26.896534 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62907a55-73e9-46d7-aa1f-1675824c165b None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:26.936466 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:26.944266 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b2515-ea3a-49e1-9d52-534f528cfa97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:26.945336 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b2515-ea3a-49e1-9d52-534f528cfa97 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:26.945627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b2515-ea3a-49e1-9d52-534f528cfa97 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:26.945909 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b2515-ea3a-49e1-9d52-534f528cfa97 None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:26.946663 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877b2515-ea3a-49e1-9d52-534f528cfa97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:26.947017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877b2515-ea3a-49e1-9d52-534f528cfa97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:26.948477 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877b2515-ea3a-49e1-9d52-534f528cfa97 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:26.966207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 602/1204] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:37:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:26.968197 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d283c86a-b5e0-4045-8f28-1ea80af31d5c None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:27.038748 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:27.101738 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d283c86a-b5e0-4045-8f28-1ea80af31d5c None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:27.147259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f018ec-4125-4120-8bd0-bae8250eeb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:27.147259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f018ec-4125-4120-8bd0-bae8250eeb95 None admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:27.147259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f018ec-4125-4120-8bd0-bae8250eeb95 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:27.147259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f018ec-4125-4120-8bd0-bae8250eeb95 None admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:27.147712 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f018ec-4125-4120-8bd0-bae8250eeb95 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:27.148725 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f018ec-4125-4120-8bd0-bae8250eeb95 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:27.173642 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 603/1205] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jul 20 14:37:26 2022] POST /identity/v3/users => generated 251 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:37:29.676724 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563b6ac-8883-44de-8f86-47e58e0c82be None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:29.677313 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563b6ac-8883-44de-8f86-47e58e0c82be None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:29.677313 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563b6ac-8883-44de-8f86-47e58e0c82be None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:29.678660 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 603/1206] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:37:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:37:29.685230 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99ce58-c781-45f3-b143-053e944dfe60 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:29.685684 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99ce58-c781-45f3-b143-053e944dfe60 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:29.685882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99ce58-c781-45f3-b143-053e944dfe60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:29.707562 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-3d99ce58-c781-45f3-b143-053e944dfe60 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:37:29.708863 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-3d99ce58-c781-45f3-b143-053e944dfe60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:37:29.787134 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 604/1207] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:37:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:37:29.813109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563b6ac-8883-44de-8f86-47e58e0c82be None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:29.953805 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563b6ac-8883-44de-8f86-47e58e0c82be None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:29.999279 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b752f2f9-40dc-4dab-9933-e8a90c831ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:30.000193 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752f2f9-40dc-4dab-9933-e8a90c831ae2 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:30.000436 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752f2f9-40dc-4dab-9933-e8a90c831ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:30.000705 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752f2f9-40dc-4dab-9933-e8a90c831ae2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:30.013534 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b752f2f9-40dc-4dab-9933-e8a90c831ae2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:37:30.022959 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-b752f2f9-40dc-4dab-9933-e8a90c831ae2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:37:30.022959 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:30.022959 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:37:30.022959 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:37:30.022959 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:37:30.022959 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:37:30.022959 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:37:30.022959 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:37:30.022959 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:37:30.022959 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:37:30.022959 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:37:30.022959 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:37:30.022959 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:37:30.022959 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:37:30.022959 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:37:30.022959 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:37:30.024183 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:37:30.024183 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:37:30.024183 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:37:30.024183 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:37:30.024183 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:37:30.024183 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:37:30.024183 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:37:30.024183 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:37:30.024183 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:37:30.024183 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:37:30.024183 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:37:30.024183 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:37:30.024183 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:37:30.024183 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:37:30.024183 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:37:30.024183 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:37:30.025263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:37:30.025263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:37:30.025263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:37:30.025263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:37:30.025263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:37:30.025263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:37:30.025263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:37:30.025263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:37:30.025263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:37:30.025263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:37:30.025263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:37:30.025263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:37:30.025263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 604/1208] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jul 20 14:37:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:37:30.030996 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99ce58-c781-45f3-b143-053e944dfe60 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:30.077928 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:30.147201 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99ce58-c781-45f3-b143-053e944dfe60 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:30.184713 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06c299-d98a-454a-8e11-0b7e6e88f67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:30.185434 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06c299-d98a-454a-8e11-0b7e6e88f67a None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:30.185721 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06c299-d98a-454a-8e11-0b7e6e88f67a None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:30.185998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06c299-d98a-454a-8e11-0b7e6e88f67a None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:30.186764 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad06c299-d98a-454a-8e11-0b7e6e88f67a None admin] RBAC: Adding query filter params (name=service) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:30.187157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad06c299-d98a-454a-8e11-0b7e6e88f67a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:30.188591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad06c299-d98a-454a-8e11-0b7e6e88f67a None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:30.197151 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 605/1209] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jul 20 14:37:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:30.203293 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b752f2f9-40dc-4dab-9933-e8a90c831ae2 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:30.274373 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:30.340508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b752f2f9-40dc-4dab-9933-e8a90c831ae2 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:30.387601 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf092386-79ad-477a-ad72-091600c95e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:30.388497 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf092386-79ad-477a-ad72-091600c95e12 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:30.388831 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf092386-79ad-477a-ad72-091600c95e12 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:30.389161 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf092386-79ad-477a-ad72-091600c95e12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:30.402024 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf092386-79ad-477a-ad72-091600c95e12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:30.404177 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf092386-79ad-477a-ad72-091600c95e12 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:30.432115 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-bf092386-79ad-477a-ad72-091600c95e12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:37:30.432115 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:30.432115 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:37:30.432115 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:37:30.432115 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:37:30.432115 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:37:30.432115 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:37:30.432115 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:37:30.432115 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:37:30.432115 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:37:30.432115 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:37:30.432115 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:37:30.432115 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:37:30.432115 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:37:30.432115 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:37:30.433149 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:37:30.433149 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:37:30.433149 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:37:30.433149 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:37:30.433149 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:37:30.433149 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:37:30.433149 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:37:30.433149 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:37:30.433149 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:37:30.433149 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:37:30.433149 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:37:30.433149 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:37:30.433149 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:37:30.433149 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:37:30.433149 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:37:30.433149 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:37:30.433149 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:37:30.434101 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:37:30.434101 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:37:30.434101 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:37:30.434101 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:37:30.434101 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 605/1210] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:37:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:37:30.444855 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06c299-d98a-454a-8e11-0b7e6e88f67a None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:30.570279 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:30.573348 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06c299-d98a-454a-8e11-0b7e6e88f67a None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:30.613160 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9573043-6e89-4383-8ee3-c04da48faf60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:30.614049 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9573043-6e89-4383-8ee3-c04da48faf60 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:30.614475 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9573043-6e89-4383-8ee3-c04da48faf60 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:30.614820 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9573043-6e89-4383-8ee3-c04da48faf60 None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:30.615639 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9573043-6e89-4383-8ee3-c04da48faf60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:30.615968 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9573043-6e89-4383-8ee3-c04da48faf60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:30.617839 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9573043-6e89-4383-8ee3-c04da48faf60 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:30.634609 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 606/1211] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:37:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:37:30.640960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf092386-79ad-477a-ad72-091600c95e12 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:30.754224 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf092386-79ad-477a-ad72-091600c95e12 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:30.757194 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:30.795550 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b4815-3fe4-4f6f-be86-4bef70f12c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:30.796523 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b4815-3fe4-4f6f-be86-4bef70f12c1b None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:30.796815 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b4815-3fe4-4f6f-be86-4bef70f12c1b None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:30.797093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b4815-3fe4-4f6f-be86-4bef70f12c1b None admin] PATH_INFO: `/v3/users/placement` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:30.805561 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683b4815-3fe4-4f6f-be86-4bef70f12c1b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:30.807235 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683b4815-3fe4-4f6f-be86-4bef70f12c1b None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:30.822627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-683b4815-3fe4-4f6f-be86-4bef70f12c1b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 14:37:30.822627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:30.822627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:37:30.822627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:37:30.822627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:37:30.822627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:37:30.822627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:37:30.822627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:37:30.822627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:37:30.822627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:37:30.822627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:37:30.822627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:37:30.822627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:37:30.822627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:37:30.822627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:37:30.822627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:37:30.824785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:37:30.824785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:37:30.824785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:37:30.824785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:37:30.824785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:37:30.824785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:37:30.824785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:37:30.824785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:37:30.824785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:37:30.824785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:37:30.824785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:37:30.824785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:37:30.824785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:37:30.824785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:37:30.824785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:37:30.824785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:37:30.824785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:37:30.824785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:37:30.825777 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:37:30.825777 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:37:30.825777 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:37:30.825777 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:37:30.825777 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:37:30.825777 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:37:30.825777 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:37:30.825777 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:37:30.825777 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 14:37:30.825777 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:37:30.825777 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 606/1212] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:37:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:37:30.830938 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9573043-6e89-4383-8ee3-c04da48faf60 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:30.944645 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9573043-6e89-4383-8ee3-c04da48faf60 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:30.990219 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c54c93-6ad3-40a1-a064-c99eac7e9398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:30.991279 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c54c93-6ad3-40a1-a064-c99eac7e9398 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:30.991579 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c54c93-6ad3-40a1-a064-c99eac7e9398 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:30.991913 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c54c93-6ad3-40a1-a064-c99eac7e9398 None admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:30.993171 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c54c93-6ad3-40a1-a064-c99eac7e9398 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:30.993490 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c54c93-6ad3-40a1-a064-c99eac7e9398 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:30.995292 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c54c93-6ad3-40a1-a064-c99eac7e9398 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:31.018529 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 607/1213] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Jul 20 14:37:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:31.024774 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b4815-3fe4-4f6f-be86-4bef70f12c1b None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:31.163406 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b4815-3fe4-4f6f-be86-4bef70f12c1b None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:31.210374 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69436a8-896a-4ed3-baba-3f8694934ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:31.211010 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69436a8-896a-4ed3-baba-3f8694934ee1 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:31.211335 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69436a8-896a-4ed3-baba-3f8694934ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:31.212503 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69436a8-896a-4ed3-baba-3f8694934ee1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:31.263554 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:31.306436 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69436a8-896a-4ed3-baba-3f8694934ee1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:31.308033 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69436a8-896a-4ed3-baba-3f8694934ee1 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:31.442898 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 607/1214] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:37:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 14:37:31.449675 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c54c93-6ad3-40a1-a064-c99eac7e9398 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:31.473368 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:31.585850 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c54c93-6ad3-40a1-a064-c99eac7e9398 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:31.641112 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a5ceee-f8cf-407e-a461-dc1e082dff43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:31.642138 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5ceee-f8cf-407e-a461-dc1e082dff43 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:31.642422 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5ceee-f8cf-407e-a461-dc1e082dff43 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:31.642772 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5ceee-f8cf-407e-a461-dc1e082dff43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:31.656653 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a5ceee-f8cf-407e-a461-dc1e082dff43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:31.658179 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a5ceee-f8cf-407e-a461-dc1e082dff43 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:31.683027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-79a5ceee-f8cf-407e-a461-dc1e082dff43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:37:31.683027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:31.683027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:37:31.683027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:37:31.683027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:37:31.683027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:37:31.683027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:37:31.683027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:37:31.683027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:37:31.683027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:37:31.683027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:37:31.683027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:37:31.683027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:37:31.683027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:37:31.683027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:37:31.684281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:37:31.684281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:37:31.684281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:37:31.684281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:37:31.684281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:37:31.684281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:37:31.684281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:37:31.684281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:37:31.684281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:37:31.684281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:37:31.684281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:37:31.684281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:37:31.684281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:37:31.684281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:37:31.684281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:37:31.684281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:37:31.684281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:37:31.690238 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:37:31.690238 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:37:31.690238 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:37:31.690238 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:37:31.690238 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 608/1215] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:37:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:37:31.692558 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69436a8-896a-4ed3-baba-3f8694934ee1 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:31.768759 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:31.817085 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69436a8-896a-4ed3-baba-3f8694934ee1 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:31.855983 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123fcc89-7a17-4948-8108-8578c4e67182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:31.856666 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fcc89-7a17-4948-8108-8578c4e67182 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:31.856951 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fcc89-7a17-4948-8108-8578c4e67182 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:31.857229 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fcc89-7a17-4948-8108-8578c4e67182 None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:31.857957 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123fcc89-7a17-4948-8108-8578c4e67182 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:31.858267 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123fcc89-7a17-4948-8108-8578c4e67182 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:31.859735 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123fcc89-7a17-4948-8108-8578c4e67182 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:31.876549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 608/1216] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:37:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:31.877889 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a5ceee-f8cf-407e-a461-dc1e082dff43 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:31.967810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:32.016581 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a5ceee-f8cf-407e-a461-dc1e082dff43 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:32.055648 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bd5b3-cc83-4fbe-8750-c9e8f5ea9b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:32.056474 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bd5b3-cc83-4fbe-8750-c9e8f5ea9b76 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:32.056760 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bd5b3-cc83-4fbe-8750-c9e8f5ea9b76 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:32.057042 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bd5b3-cc83-4fbe-8750-c9e8f5ea9b76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:32.064224 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531bd5b3-cc83-4fbe-8750-c9e8f5ea9b76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:32.065833 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531bd5b3-cc83-4fbe-8750-c9e8f5ea9b76 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:32.080200 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-531bd5b3-cc83-4fbe-8750-c9e8f5ea9b76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:37:32.080200 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:32.080200 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:37:32.080200 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:37:32.080200 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:37:32.080200 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:37:32.080200 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:37:32.080200 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:37:32.080200 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:37:32.080200 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:37:32.080200 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:37:32.080200 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:37:32.080200 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:37:32.080200 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:37:32.080200 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:37:32.081341 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:37:32.081341 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:37:32.081341 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:37:32.081341 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:37:32.081341 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:37:32.081341 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:37:32.081341 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:37:32.081341 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:37:32.081341 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:37:32.081341 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:37:32.081341 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:37:32.081341 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:37:32.081341 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:37:32.081341 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:37:32.081341 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:37:32.081341 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:37:32.081341 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:37:32.082541 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:37:32.082541 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:37:32.082541 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:37:32.082541 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:37:32.082541 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:37:32.082541 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:37:32.082541 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:37:32.082541 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:37:32.082541 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 609/1217] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:37:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:37:32.087853 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123fcc89-7a17-4948-8108-8578c4e67182 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:32.195750 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123fcc89-7a17-4948-8108-8578c4e67182 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:32.224597 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:32.246943 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee116b6-e3cf-4d97-952e-c1160e4a33c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:32.247919 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee116b6-e3cf-4d97-952e-c1160e4a33c1 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:32.248354 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee116b6-e3cf-4d97-952e-c1160e4a33c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:32.248800 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee116b6-e3cf-4d97-952e-c1160e4a33c1 None admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:32.249963 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee116b6-e3cf-4d97-952e-c1160e4a33c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:32.250496 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee116b6-e3cf-4d97-952e-c1160e4a33c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:32.252679 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee116b6-e3cf-4d97-952e-c1160e4a33c1 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:32.267090 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 609/1218] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:37:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:32.274169 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bd5b3-cc83-4fbe-8750-c9e8f5ea9b76 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:32.410622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bd5b3-cc83-4fbe-8750-c9e8f5ea9b76 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:32.445271 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:32.451928 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408056aa-ec80-499c-91f3-d02770ee664f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:32.452662 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408056aa-ec80-499c-91f3-d02770ee664f None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:32.452951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408056aa-ec80-499c-91f3-d02770ee664f None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:32.453341 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408056aa-ec80-499c-91f3-d02770ee664f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:32.454387 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408056aa-ec80-499c-91f3-d02770ee664f None admin] RBAC: Adding query filter params (role.id=62e9fa78a7c44a16bb4540d3e8d7044d, scope.project.id=59333e4f593346e184877d6bd9d22026, user.id=cb857f32bbc04d82bfa681bd5659765f) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:32.454827 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408056aa-ec80-499c-91f3-d02770ee664f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:32.456307 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408056aa-ec80-499c-91f3-d02770ee664f None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:32.473176 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 610/1219] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:37:32 2022] GET /identity/v3/role_assignments?user.id=cb857f32bbc04d82bfa681bd5659765f&scope.project.id=59333e4f593346e184877d6bd9d22026&role.id=62e9fa78a7c44a16bb4540d3e8d7044d => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:34.730834 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb7bf2-ddac-4b8c-8b17-f0c0ae36f429 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:34.731421 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb7bf2-ddac-4b8c-8b17-f0c0ae36f429 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:34.731421 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb7bf2-ddac-4b8c-8b17-f0c0ae36f429 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:34.733127 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 610/1220] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:37:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:37:34.739143 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e620ff-4930-4d42-9c89-7d03e1d36d7f None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:34.739656 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e620ff-4930-4d42-9c89-7d03e1d36d7f None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:34.739752 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e620ff-4930-4d42-9c89-7d03e1d36d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:34.756926 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-44e620ff-4930-4d42-9c89-7d03e1d36d7f None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:37:34.758145 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-44e620ff-4930-4d42-9c89-7d03e1d36d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:37:34.841577 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 611/1221] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:37:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:37:34.864886 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb7bf2-ddac-4b8c-8b17-f0c0ae36f429 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:34.971567 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb7bf2-ddac-4b8c-8b17-f0c0ae36f429 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:35.008541 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c18fa-7120-4db9-9779-3058a685529e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:35.009457 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c18fa-7120-4db9-9779-3058a685529e None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:35.009771 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c18fa-7120-4db9-9779-3058a685529e None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:35.010083 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c18fa-7120-4db9-9779-3058a685529e None admin] PATH_INFO: `/v3/roles/service` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:35.021850 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c18fa-7120-4db9-9779-3058a685529e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:37:35.029529 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-c88c18fa-7120-4db9-9779-3058a685529e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:37:35.029529 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:35.029529 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:37:35.029529 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:37:35.029529 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:37:35.029529 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:37:35.029529 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:37:35.029529 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:37:35.029529 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:37:35.029529 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:37:35.029529 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:37:35.029529 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:37:35.029529 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:37:35.029529 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:37:35.029529 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:37:35.029529 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:37:35.030798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:37:35.030798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:37:35.030798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:37:35.030798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:37:35.030798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:37:35.030798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:37:35.030798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:37:35.030798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:37:35.030798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:37:35.030798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:37:35.030798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:37:35.030798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:37:35.030798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:37:35.030798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:37:35.030798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:37:35.030798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:37:35.031739 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:37:35.031739 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:37:35.031739 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:37:35.031739 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:37:35.031739 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:37:35.031739 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:37:35.031739 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:37:35.031739 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:37:35.031739 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:37:35.031739 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:37:35.031739 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:37:35.031739 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:37:35.031739 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 611/1222] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jul 20 14:37:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:37:35.037054 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e620ff-4930-4d42-9c89-7d03e1d36d7f None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:35.143876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e620ff-4930-4d42-9c89-7d03e1d36d7f None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:35.168517 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:35.179555 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c3fa0-e625-45a6-bd4a-5337103ed9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:35.180333 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c3fa0-e625-45a6-bd4a-5337103ed9e5 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:35.180522 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c3fa0-e625-45a6-bd4a-5337103ed9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:35.180798 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c3fa0-e625-45a6-bd4a-5337103ed9e5 None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:35.181550 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084c3fa0-e625-45a6-bd4a-5337103ed9e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:35.181854 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084c3fa0-e625-45a6-bd4a-5337103ed9e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:35.183515 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084c3fa0-e625-45a6-bd4a-5337103ed9e5 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:35.193809 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 612/1223] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jul 20 14:37:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:35.200018 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c18fa-7120-4db9-9779-3058a685529e None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:35.249317 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:35.308911 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c18fa-7120-4db9-9779-3058a685529e None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:35.346067 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf5172-8315-44a0-960c-6c0d29ee23c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:35.347139 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf5172-8315-44a0-960c-6c0d29ee23c8 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:35.347412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf5172-8315-44a0-960c-6c0d29ee23c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:35.347808 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf5172-8315-44a0-960c-6c0d29ee23c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:35.359245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf5172-8315-44a0-960c-6c0d29ee23c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:35.360715 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf5172-8315-44a0-960c-6c0d29ee23c8 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:35.380562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-febf5172-8315-44a0-960c-6c0d29ee23c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:37:35.380562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:35.380562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:37:35.380562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:37:35.380562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:37:35.380562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:37:35.380562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:37:35.380562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:37:35.380562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:37:35.380562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:37:35.380562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:37:35.380562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:37:35.380562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:37:35.380562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:37:35.380562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:37:35.381549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:37:35.381549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:37:35.381549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:37:35.381549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:37:35.381549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:37:35.381549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:37:35.381549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:37:35.381549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:37:35.381549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:37:35.381549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:37:35.381549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:37:35.381549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:37:35.381549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:37:35.381549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:37:35.381549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:37:35.381549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:37:35.381549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:37:35.388014 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:37:35.388014 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:37:35.388014 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:37:35.388014 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:37:35.388014 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 612/1224] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:37:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:37:35.388354 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c3fa0-e625-45a6-bd4a-5337103ed9e5 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:35.498259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c3fa0-e625-45a6-bd4a-5337103ed9e5 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:35.534275 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2099635-1bab-4cb4-800d-1a3c31baf2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:35.534979 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2099635-1bab-4cb4-800d-1a3c31baf2f3 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:35.535410 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2099635-1bab-4cb4-800d-1a3c31baf2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:35.535801 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2099635-1bab-4cb4-800d-1a3c31baf2f3 None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:35.536653 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2099635-1bab-4cb4-800d-1a3c31baf2f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:35.536927 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2099635-1bab-4cb4-800d-1a3c31baf2f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:35.538376 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2099635-1bab-4cb4-800d-1a3c31baf2f3 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:35.555830 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 613/1225] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:37:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:35.557013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf5172-8315-44a0-960c-6c0d29ee23c8 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:35.654807 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:35.662418 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf5172-8315-44a0-960c-6c0d29ee23c8 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:35.697738 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e179e-dc07-4af1-b101-9b8a53d75df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:35.698564 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e179e-dc07-4af1-b101-9b8a53d75df0 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:35.698833 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e179e-dc07-4af1-b101-9b8a53d75df0 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:35.699129 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e179e-dc07-4af1-b101-9b8a53d75df0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:35.707198 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e179e-dc07-4af1-b101-9b8a53d75df0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:35.708684 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e179e-dc07-4af1-b101-9b8a53d75df0 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:35.724092 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-276e179e-dc07-4af1-b101-9b8a53d75df0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 14:37:35.724092 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:35.724092 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:37:35.724092 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:37:35.724092 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:37:35.724092 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:37:35.724092 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:37:35.724092 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:37:35.724092 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:37:35.724092 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:37:35.724092 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:37:35.724092 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:37:35.724092 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:37:35.724092 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:37:35.724092 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:37:35.724092 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:37:35.725268 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:37:35.725268 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:37:35.725268 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:37:35.725268 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:37:35.725268 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:37:35.725268 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:37:35.725268 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:37:35.725268 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:37:35.725268 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:37:35.725268 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:37:35.725268 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:37:35.725268 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:37:35.725268 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:37:35.725268 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:37:35.725268 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:37:35.725268 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:37:35.725268 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:37:35.725268 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:37:35.731715 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:37:35.731715 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:37:35.731715 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:37:35.731715 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:37:35.731715 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:37:35.731715 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:37:35.731715 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:37:35.731715 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:37:35.731715 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 14:37:35.731715 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:37:35.731715 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 613/1226] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:37:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:37:35.732456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2099635-1bab-4cb4-800d-1a3c31baf2f3 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:35.781064 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:35.845297 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2099635-1bab-4cb4-800d-1a3c31baf2f3 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:35.882037 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90e7b6-9f49-4651-af04-dd5ecf19bded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:35.882747 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90e7b6-9f49-4651-af04-dd5ecf19bded None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:35.883150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90e7b6-9f49-4651-af04-dd5ecf19bded None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:35.883339 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90e7b6-9f49-4651-af04-dd5ecf19bded None admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:35.884170 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c90e7b6-9f49-4651-af04-dd5ecf19bded None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:35.884479 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c90e7b6-9f49-4651-af04-dd5ecf19bded None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:35.885969 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c90e7b6-9f49-4651-af04-dd5ecf19bded None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:35.904591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 614/1227] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Jul 20 14:37:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:35.910661 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e179e-dc07-4af1-b101-9b8a53d75df0 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:36.018257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e179e-dc07-4af1-b101-9b8a53d75df0 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:36.056893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5581f1-21ed-4f86-8027-53965d0bc8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:36.057698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5581f1-21ed-4f86-8027-53965d0bc8ee None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:36.057980 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5581f1-21ed-4f86-8027-53965d0bc8ee None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:36.058251 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5581f1-21ed-4f86-8027-53965d0bc8ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:36.071815 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5581f1-21ed-4f86-8027-53965d0bc8ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:36.073788 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5581f1-21ed-4f86-8027-53965d0bc8ee None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:36.095751 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-fe5581f1-21ed-4f86-8027-53965d0bc8ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:37:36.095751 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:36.095751 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:37:36.095751 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:37:36.095751 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:37:36.095751 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:37:36.095751 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:37:36.095751 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:37:36.095751 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:37:36.095751 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:37:36.095751 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:37:36.095751 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:37:36.095751 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:37:36.095751 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:37:36.095751 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:37:36.096924 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:37:36.096924 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:37:36.096924 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:37:36.096924 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:37:36.096924 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:37:36.096924 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:37:36.096924 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:37:36.096924 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:37:36.096924 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:37:36.096924 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:37:36.096924 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:37:36.096924 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:37:36.096924 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:37:36.096924 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:37:36.096924 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:37:36.096924 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:37:36.096924 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:37:36.097896 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:37:36.097896 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:37:36.097896 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:37:36.097896 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:37:36.097896 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 614/1228] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:37:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:37:36.103111 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90e7b6-9f49-4651-af04-dd5ecf19bded None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:36.209298 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90e7b6-9f49-4651-af04-dd5ecf19bded None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:36.211694 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:36.254036 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d907b-9f83-4481-b1c6-57e4b23dd24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:36.255231 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d907b-9f83-4481-b1c6-57e4b23dd24a None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:36.255630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d907b-9f83-4481-b1c6-57e4b23dd24a None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:36.256223 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d907b-9f83-4481-b1c6-57e4b23dd24a None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:36.257316 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8d907b-9f83-4481-b1c6-57e4b23dd24a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:36.257846 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8d907b-9f83-4481-b1c6-57e4b23dd24a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:36.260006 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8d907b-9f83-4481-b1c6-57e4b23dd24a None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:36.274680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 615/1229] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:37:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:36.281019 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5581f1-21ed-4f86-8027-53965d0bc8ee None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:36.291970 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:36.414009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5581f1-21ed-4f86-8027-53965d0bc8ee None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:36.459944 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e2999-d2fb-47e9-a374-3753b53a8a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:36.461244 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e2999-d2fb-47e9-a374-3753b53a8a92 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:36.461737 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e2999-d2fb-47e9-a374-3753b53a8a92 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:36.462180 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e2999-d2fb-47e9-a374-3753b53a8a92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:36.470694 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3e2999-d2fb-47e9-a374-3753b53a8a92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:36.472220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3e2999-d2fb-47e9-a374-3753b53a8a92 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:36.489358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-3a3e2999-d2fb-47e9-a374-3753b53a8a92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:37:36.489358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:36.489358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:37:36.489358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:37:36.489358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:37:36.489358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:37:36.489358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:37:36.489358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:37:36.489358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:37:36.489358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:37:36.489358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:37:36.489358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:37:36.489358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:37:36.489358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:37:36.489358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:37:36.490421 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:37:36.490421 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:37:36.490421 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:37:36.490421 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:37:36.490421 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:37:36.490421 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:37:36.490421 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:37:36.490421 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:37:36.490421 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:37:36.490421 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:37:36.490421 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:37:36.490421 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:37:36.490421 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:37:36.490421 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:37:36.490421 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:37:36.490421 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:37:36.490421 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:37:36.491550 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:37:36.491550 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:37:36.491550 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:37:36.491550 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:37:36.491550 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:37:36.491550 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:37:36.491550 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:37:36.491550 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:37:36.491550 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 615/1230] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:37:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:37:36.496713 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d907b-9f83-4481-b1c6-57e4b23dd24a None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:36.612310 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d907b-9f83-4481-b1c6-57e4b23dd24a None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:36.650049 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd58cf5-40e8-4da5-912a-5af203220284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:36.650712 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd58cf5-40e8-4da5-912a-5af203220284 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:36.650996 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd58cf5-40e8-4da5-912a-5af203220284 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:36.651301 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd58cf5-40e8-4da5-912a-5af203220284 None admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:36.652145 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd58cf5-40e8-4da5-912a-5af203220284 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:36.652431 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd58cf5-40e8-4da5-912a-5af203220284 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:36.653841 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd58cf5-40e8-4da5-912a-5af203220284 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:36.666495 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 616/1231] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:37:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:36.673192 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e2999-d2fb-47e9-a374-3753b53a8a92 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:36.747115 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:36.786795 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e2999-d2fb-47e9-a374-3753b53a8a92 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:36.827972 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a54e7b-4363-420f-ac5f-1254a533e225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:36.828992 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a54e7b-4363-420f-ac5f-1254a533e225 None admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:36.829277 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a54e7b-4363-420f-ac5f-1254a533e225 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:36.829629 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a54e7b-4363-420f-ac5f-1254a533e225 None admin] PATH_INFO: `/v3/projects/59333e4f593346e184877d6bd9d22026/users/cb857f32bbc04d82bfa681bd5659765f/roles/62e9fa78a7c44a16bb4540d3e8d7044d` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:36.845181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a54e7b-4363-420f-ac5f-1254a533e225 None admin] RBAC: Authorizing `identity:create_grant(project_id=59333e4f593346e184877d6bd9d22026, user_id=cb857f32bbc04d82bfa681bd5659765f, role_id=62e9fa78a7c44a16bb4540d3e8d7044d)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:36.846686 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a54e7b-4363-420f-ac5f-1254a533e225 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:36.861764 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 616/1232] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jul 20 14:37:36 2022] PUT /identity/v3/projects/59333e4f593346e184877d6bd9d22026/users/cb857f32bbc04d82bfa681bd5659765f/roles/62e9fa78a7c44a16bb4540d3e8d7044d => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:37:38.979020 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f7089-3e66-45e1-b72d-a76639cf0ae1 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:38.979592 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f7089-3e66-45e1-b72d-a76639cf0ae1 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:38.979685 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f7089-3e66-45e1-b72d-a76639cf0ae1 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:38.981158 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 617/1233] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:37:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:37:38.986565 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da70025-34e4-4360-99f6-99d8eccbbb9b None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:38.986999 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da70025-34e4-4360-99f6-99d8eccbbb9b None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:38.987170 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da70025-34e4-4360-99f6-99d8eccbbb9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:39.006768 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-3da70025-34e4-4360-99f6-99d8eccbbb9b None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:37:39.008417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-3da70025-34e4-4360-99f6-99d8eccbbb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:37:39.080999 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 617/1234] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:37:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:37:39.104762 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730f7089-3e66-45e1-b72d-a76639cf0ae1 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:39.131647 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:39.211822 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730f7089-3e66-45e1-b72d-a76639cf0ae1 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:39.256001 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a2a93-a3a0-400a-9224-728e5a8ff833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:39.256819 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a2a93-a3a0-400a-9224-728e5a8ff833 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:39.257185 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a2a93-a3a0-400a-9224-728e5a8ff833 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:39.257476 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a2a93-a3a0-400a-9224-728e5a8ff833 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:39.270761 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766a2a93-a3a0-400a-9224-728e5a8ff833 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:37:39.279985 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-766a2a93-a3a0-400a-9224-728e5a8ff833 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:37:39.279985 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:39.279985 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:37:39.279985 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:37:39.279985 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:37:39.279985 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:37:39.279985 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:37:39.279985 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:37:39.279985 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:37:39.279985 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:37:39.279985 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:37:39.279985 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:37:39.279985 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:37:39.279985 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:37:39.279985 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:37:39.279985 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:37:39.281304 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:37:39.281304 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:37:39.281304 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:37:39.281304 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:37:39.281304 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:37:39.281304 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:37:39.281304 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:37:39.281304 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:37:39.281304 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:37:39.281304 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:37:39.281304 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:37:39.281304 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:37:39.281304 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:37:39.281304 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:37:39.281304 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:37:39.281304 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:37:39.287551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:37:39.287551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:37:39.287551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:37:39.287551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:37:39.287551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:37:39.287551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:37:39.287551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:37:39.287551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:37:39.287551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:37:39.287551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:37:39.287551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 14:37:39.287551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:37:39.287551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 618/1235] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jul 20 14:37:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:37:39.288279 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da70025-34e4-4360-99f6-99d8eccbbb9b None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:39.341006 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:39.401536 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da70025-34e4-4360-99f6-99d8eccbbb9b None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:39.442564 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5006c8-9d9a-4a8d-8c2c-47a237db293a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:39.443371 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5006c8-9d9a-4a8d-8c2c-47a237db293a None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:39.443790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5006c8-9d9a-4a8d-8c2c-47a237db293a None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:39.444234 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5006c8-9d9a-4a8d-8c2c-47a237db293a None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:39.445147 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5006c8-9d9a-4a8d-8c2c-47a237db293a None admin] RBAC: Adding query filter params (name=service) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:39.445668 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5006c8-9d9a-4a8d-8c2c-47a237db293a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:39.447516 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5006c8-9d9a-4a8d-8c2c-47a237db293a None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:39.457333 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 618/1236] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jul 20 14:37:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:39.463195 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a2a93-a3a0-400a-9224-728e5a8ff833 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:39.559412 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:39.577285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a2a93-a3a0-400a-9224-728e5a8ff833 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:39.618232 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11eeb6d-3046-45a4-a14b-f1f255b37d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:39.619039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11eeb6d-3046-45a4-a14b-f1f255b37d24 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:39.619358 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11eeb6d-3046-45a4-a14b-f1f255b37d24 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:39.619693 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11eeb6d-3046-45a4-a14b-f1f255b37d24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:39.630677 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11eeb6d-3046-45a4-a14b-f1f255b37d24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:39.632192 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11eeb6d-3046-45a4-a14b-f1f255b37d24 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:39.650630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-f11eeb6d-3046-45a4-a14b-f1f255b37d24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:37:39.650630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:39.650630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:37:39.650630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:37:39.650630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:37:39.650630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:37:39.650630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:37:39.650630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:37:39.650630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:37:39.650630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:37:39.650630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:37:39.650630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:37:39.650630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:37:39.650630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:37:39.650630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:37:39.651814 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:37:39.651814 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:37:39.651814 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:37:39.651814 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:37:39.651814 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:37:39.651814 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:37:39.651814 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:37:39.651814 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:37:39.651814 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:37:39.651814 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:37:39.651814 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:37:39.651814 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:37:39.651814 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:37:39.651814 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:37:39.651814 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:37:39.651814 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:37:39.651814 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:37:39.652941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:37:39.652941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:37:39.652941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:37:39.652941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:37:39.652941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 619/1237] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:37:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:37:39.657853 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5006c8-9d9a-4a8d-8c2c-47a237db293a None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:39.767434 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5006c8-9d9a-4a8d-8c2c-47a237db293a None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:39.777880 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:39.805323 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b2350-2a56-4d04-b048-2c0833333704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:39.806024 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b2350-2a56-4d04-b048-2c0833333704 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:39.806305 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b2350-2a56-4d04-b048-2c0833333704 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:39.806576 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b2350-2a56-4d04-b048-2c0833333704 None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:39.807335 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574b2350-2a56-4d04-b048-2c0833333704 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:39.807641 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574b2350-2a56-4d04-b048-2c0833333704 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:39.809021 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574b2350-2a56-4d04-b048-2c0833333704 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:39.823813 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 619/1238] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:39.830025 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11eeb6d-3046-45a4-a14b-f1f255b37d24 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:39.938354 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11eeb6d-3046-45a4-a14b-f1f255b37d24 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:39.975023 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d819f9a-4494-41c8-a8ad-7058f763afc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:39.975909 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d819f9a-4494-41c8-a8ad-7058f763afc2 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:39.976190 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d819f9a-4494-41c8-a8ad-7058f763afc2 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:39.976461 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d819f9a-4494-41c8-a8ad-7058f763afc2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:39.979325 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:39.985329 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d819f9a-4494-41c8-a8ad-7058f763afc2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:39.986969 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d819f9a-4494-41c8-a8ad-7058f763afc2 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:40.003871 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-9d819f9a-4494-41c8-a8ad-7058f763afc2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 14:37:40.003871 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:40.003871 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:37:40.003871 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:37:40.003871 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:37:40.003871 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:37:40.003871 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:37:40.003871 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:37:40.003871 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:37:40.003871 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:37:40.003871 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:37:40.003871 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:37:40.003871 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:37:40.003871 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:37:40.003871 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:37:40.003871 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:37:40.005088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:37:40.005088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:37:40.005088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:37:40.005088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:37:40.005088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:37:40.005088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:37:40.005088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:37:40.005088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:37:40.005088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:37:40.005088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:37:40.005088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:37:40.005088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:37:40.005088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:37:40.005088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:37:40.005088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:37:40.005088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:37:40.005088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:37:40.005088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:37:40.006418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:37:40.006418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:37:40.006418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:37:40.006418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:37:40.006418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:37:40.006418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:37:40.006418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:37:40.006418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:37:40.006418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 14:37:40.006418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:37:40.006418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 620/1239] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:37:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:37:40.011559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b2350-2a56-4d04-b048-2c0833333704 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:40.130308 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b2350-2a56-4d04-b048-2c0833333704 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:40.166837 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2b79f-7de6-4349-a796-160fa6a748c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:40.167654 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2b79f-7de6-4349-a796-160fa6a748c0 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:40.168283 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2b79f-7de6-4349-a796-160fa6a748c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:40.168629 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2b79f-7de6-4349-a796-160fa6a748c0 None admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:40.169642 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2b79f-7de6-4349-a796-160fa6a748c0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:40.169944 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2b79f-7de6-4349-a796-160fa6a748c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:40.171663 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2b79f-7de6-4349-a796-160fa6a748c0 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:40.189884 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 620/1240] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Jul 20 14:37:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:40.196028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d819f9a-4494-41c8-a8ad-7058f763afc2 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:40.314583 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d819f9a-4494-41c8-a8ad-7058f763afc2 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:40.352415 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67469407-28e4-4e7c-964d-a8d09512d868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:40.353096 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67469407-28e4-4e7c-964d-a8d09512d868 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:40.353389 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67469407-28e4-4e7c-964d-a8d09512d868 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:40.353663 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67469407-28e4-4e7c-964d-a8d09512d868 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:40.426274 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:40.434819 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67469407-28e4-4e7c-964d-a8d09512d868 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:40.436539 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67469407-28e4-4e7c-964d-a8d09512d868 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:40.549362 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 621/1241] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:37:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 14:37:40.550791 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2b79f-7de6-4349-a796-160fa6a748c0 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:40.578569 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:40.668396 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2b79f-7de6-4349-a796-160fa6a748c0 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:40.704722 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f3a27-d64c-4746-be2b-87fe576d96fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:40.705458 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f3a27-d64c-4746-be2b-87fe576d96fd None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:40.705797 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f3a27-d64c-4746-be2b-87fe576d96fd None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:40.706071 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f3a27-d64c-4746-be2b-87fe576d96fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:40.717435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f3a27-d64c-4746-be2b-87fe576d96fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:40.719303 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f3a27-d64c-4746-be2b-87fe576d96fd None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:40.742369 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-505f3a27-d64c-4746-be2b-87fe576d96fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:37:40.742369 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:40.742369 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:37:40.742369 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:37:40.742369 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:37:40.742369 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:37:40.742369 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:37:40.742369 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:37:40.742369 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:37:40.742369 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:37:40.742369 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:37:40.742369 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:37:40.742369 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:37:40.742369 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:37:40.742369 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:37:40.743559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:37:40.743559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:37:40.743559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:37:40.743559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:37:40.743559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:37:40.743559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:37:40.743559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:37:40.743559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:37:40.743559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:37:40.743559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:37:40.743559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:37:40.743559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:37:40.743559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:37:40.743559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:37:40.743559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:37:40.743559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:37:40.743559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:37:40.744726 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:37:40.744726 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:37:40.744726 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:37:40.744726 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:37:40.744726 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 621/1242] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:37:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:37:40.751007 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67469407-28e4-4e7c-964d-a8d09512d868 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:40.855817 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:40.863823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67469407-28e4-4e7c-964d-a8d09512d868 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:40.903363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec4ef4-8a18-4a81-a6bd-c76816137c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:40.904106 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec4ef4-8a18-4a81-a6bd-c76816137c81 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:40.904412 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec4ef4-8a18-4a81-a6bd-c76816137c81 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:40.904682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec4ef4-8a18-4a81-a6bd-c76816137c81 None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:40.905505 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cec4ef4-8a18-4a81-a6bd-c76816137c81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:40.905810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cec4ef4-8a18-4a81-a6bd-c76816137c81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:40.907356 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cec4ef4-8a18-4a81-a6bd-c76816137c81 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:40.921858 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 622/1243] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:37:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:40.928761 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f3a27-d64c-4746-be2b-87fe576d96fd None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:41.024904 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:41.048945 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f3a27-d64c-4746-be2b-87fe576d96fd None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:41.086796 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96178f2-9c5a-438b-8cc2-1b4922a73469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:41.087651 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96178f2-9c5a-438b-8cc2-1b4922a73469 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:41.087985 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96178f2-9c5a-438b-8cc2-1b4922a73469 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:41.088343 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96178f2-9c5a-438b-8cc2-1b4922a73469 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:41.095584 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96178f2-9c5a-438b-8cc2-1b4922a73469 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:41.097085 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96178f2-9c5a-438b-8cc2-1b4922a73469 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:41.110754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-e96178f2-9c5a-438b-8cc2-1b4922a73469 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:37:41.110754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:41.110754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:37:41.110754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:37:41.110754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:37:41.110754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:37:41.110754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:37:41.110754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:37:41.110754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:37:41.110754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:37:41.110754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:37:41.110754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:37:41.110754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:37:41.110754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:37:41.110754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:37:41.111996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:37:41.111996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:37:41.111996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:37:41.111996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:37:41.111996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:37:41.111996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:37:41.111996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:37:41.111996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:37:41.111996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:37:41.111996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:37:41.111996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:37:41.111996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:37:41.111996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:37:41.111996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:37:41.111996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:37:41.111996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:37:41.111996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:37:41.113012 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:37:41.113012 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:37:41.113012 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:37:41.113012 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:37:41.113012 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:37:41.113012 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:37:41.113012 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:37:41.113012 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:37:41.113012 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 622/1244] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:37:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:37:41.118339 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec4ef4-8a18-4a81-a6bd-c76816137c81 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:41.221851 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec4ef4-8a18-4a81-a6bd-c76816137c81 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:41.260009 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d45a4-3361-4ccc-82dd-2dd9b3d22beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:41.260693 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d45a4-3361-4ccc-82dd-2dd9b3d22beb None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:41.260969 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d45a4-3361-4ccc-82dd-2dd9b3d22beb None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:41.261246 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d45a4-3361-4ccc-82dd-2dd9b3d22beb None admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:41.262065 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3d45a4-3361-4ccc-82dd-2dd9b3d22beb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:41.262352 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3d45a4-3361-4ccc-82dd-2dd9b3d22beb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:41.263923 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3d45a4-3361-4ccc-82dd-2dd9b3d22beb None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:41.278110 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 623/1245] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:37:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:41.285747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96178f2-9c5a-438b-8cc2-1b4922a73469 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:41.393320 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96178f2-9c5a-438b-8cc2-1b4922a73469 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:41.429918 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd14d16-39a1-40a7-baeb-47a27cb94275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:41.429918 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd14d16-39a1-40a7-baeb-47a27cb94275 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:41.430303 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd14d16-39a1-40a7-baeb-47a27cb94275 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:41.430488 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd14d16-39a1-40a7-baeb-47a27cb94275 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:41.431354 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd14d16-39a1-40a7-baeb-47a27cb94275 None admin] RBAC: Adding query filter params (role.id=62e9fa78a7c44a16bb4540d3e8d7044d, scope.project.id=59333e4f593346e184877d6bd9d22026, user.id=cb857f32bbc04d82bfa681bd5659765f) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:41.431660 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd14d16-39a1-40a7-baeb-47a27cb94275 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:41.433234 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd14d16-39a1-40a7-baeb-47a27cb94275 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:41.439952 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 623/1246] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:37:41 2022] GET /identity/v3/role_assignments?user.id=cb857f32bbc04d82bfa681bd5659765f&scope.project.id=59333e4f593346e184877d6bd9d22026&role.id=62e9fa78a7c44a16bb4540d3e8d7044d => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:43.547406 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a9100-ab66-4dbf-9a29-f8bce55bbab4 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:43.548042 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a9100-ab66-4dbf-9a29-f8bce55bbab4 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:43.548042 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a9100-ab66-4dbf-9a29-f8bce55bbab4 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:43.549330 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 624/1247] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:37:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:37:43.554860 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b841b-7d32-4196-bd32-ebd62e663ad0 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:43.555363 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b841b-7d32-4196-bd32-ebd62e663ad0 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:43.555461 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b841b-7d32-4196-bd32-ebd62e663ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:43.561538 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:43.576333 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-b92b841b-7d32-4196-bd32-ebd62e663ad0 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:37:43.577729 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-b92b841b-7d32-4196-bd32-ebd62e663ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:37:43.660358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 624/1248] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:37:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:37:43.684820 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a9100-ab66-4dbf-9a29-f8bce55bbab4 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:43.706505 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:43.800341 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a9100-ab66-4dbf-9a29-f8bce55bbab4 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:43.839865 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee6d9d-eb5a-4331-934b-2c78df60e57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:43.840822 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee6d9d-eb5a-4331-934b-2c78df60e57d None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:43.841117 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee6d9d-eb5a-4331-934b-2c78df60e57d None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:43.841413 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee6d9d-eb5a-4331-934b-2c78df60e57d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:43.853614 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ee6d9d-eb5a-4331-934b-2c78df60e57d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:37:43.861354 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-55ee6d9d-eb5a-4331-934b-2c78df60e57d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:37:43.861354 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:43.861354 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:37:43.861354 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:37:43.861354 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:37:43.861354 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:37:43.861354 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:37:43.861354 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:37:43.861354 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:37:43.861354 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:37:43.861354 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:37:43.861354 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:37:43.861354 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:37:43.861354 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:37:43.861354 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:37:43.861354 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:37:43.862465 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:37:43.862465 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:37:43.862465 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:37:43.862465 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:37:43.862465 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:37:43.862465 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:37:43.862465 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:37:43.862465 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:37:43.862465 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:37:43.862465 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:37:43.862465 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:37:43.862465 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:37:43.862465 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:37:43.862465 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:37:43.862465 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:37:43.862465 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:37:43.863476 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:37:43.863476 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:37:43.863476 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:37:43.863476 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:37:43.863476 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:37:43.863476 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:37:43.863476 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:37:43.863476 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:37:43.863476 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:37:43.863476 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:37:43.863476 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:37:43.863476 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:37:43.863476 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 625/1249] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:37:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 14:37:43.869382 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b841b-7d32-4196-bd32-ebd62e663ad0 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:43.985828 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b841b-7d32-4196-bd32-ebd62e663ad0 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:44.020786 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:44.026573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ed497-355e-4f60-af21-3b37cef8e112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:44.027311 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ed497-355e-4f60-af21-3b37cef8e112 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:44.027451 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ed497-355e-4f60-af21-3b37cef8e112 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:44.027712 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ed497-355e-4f60-af21-3b37cef8e112 None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:44.028473 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127ed497-355e-4f60-af21-3b37cef8e112 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:44.028791 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127ed497-355e-4f60-af21-3b37cef8e112 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:44.030306 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127ed497-355e-4f60-af21-3b37cef8e112 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:44.039930 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 625/1250] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:37:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:44.048567 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee6d9d-eb5a-4331-934b-2c78df60e57d None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:44.132427 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:44.157989 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee6d9d-eb5a-4331-934b-2c78df60e57d None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:44.194636 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139cba26-80d2-4e78-aaa4-095bbfb7ec51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:44.195561 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cba26-80d2-4e78-aaa4-095bbfb7ec51 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:44.196016 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cba26-80d2-4e78-aaa4-095bbfb7ec51 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:44.196361 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cba26-80d2-4e78-aaa4-095bbfb7ec51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:44.211096 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139cba26-80d2-4e78-aaa4-095bbfb7ec51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:44.212616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139cba26-80d2-4e78-aaa4-095bbfb7ec51 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:44.234775 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-139cba26-80d2-4e78-aaa4-095bbfb7ec51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:37:44.234775 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:44.234775 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:37:44.234775 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:37:44.234775 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:37:44.234775 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:37:44.234775 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:37:44.234775 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:37:44.234775 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:37:44.234775 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:37:44.234775 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:37:44.234775 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:37:44.234775 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:37:44.234775 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:37:44.234775 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:37:44.235880 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:37:44.235880 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:37:44.235880 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:37:44.235880 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:37:44.235880 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:37:44.235880 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:37:44.235880 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:37:44.235880 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:37:44.235880 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:37:44.235880 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:37:44.235880 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:37:44.235880 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:37:44.235880 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:37:44.235880 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:37:44.235880 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:37:44.235880 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:37:44.235880 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:37:44.236786 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:37:44.236786 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:37:44.236786 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:37:44.236786 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:37:44.236786 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 626/1251] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:37:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:37:44.242936 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ed497-355e-4f60-af21-3b37cef8e112 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:44.370963 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ed497-355e-4f60-af21-3b37cef8e112 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:44.414166 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e673fceb-9d49-493d-94a9-f95bc1adce63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:44.414762 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673fceb-9d49-493d-94a9-f95bc1adce63 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:44.415103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673fceb-9d49-493d-94a9-f95bc1adce63 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:44.415381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673fceb-9d49-493d-94a9-f95bc1adce63 None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:44.416111 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e673fceb-9d49-493d-94a9-f95bc1adce63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:44.416419 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e673fceb-9d49-493d-94a9-f95bc1adce63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:44.418147 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e673fceb-9d49-493d-94a9-f95bc1adce63 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:44.431042 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 626/1252] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:37:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:44.437243 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139cba26-80d2-4e78-aaa4-095bbfb7ec51 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:44.553228 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139cba26-80d2-4e78-aaa4-095bbfb7ec51 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:44.584844 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:44.594224 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afe505-35a8-4590-8cd4-eaf9a1d3f0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:44.595209 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afe505-35a8-4590-8cd4-eaf9a1d3f0e1 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:44.595548 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afe505-35a8-4590-8cd4-eaf9a1d3f0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:44.595887 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afe505-35a8-4590-8cd4-eaf9a1d3f0e1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:44.604674 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06afe505-35a8-4590-8cd4-eaf9a1d3f0e1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:44.606236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06afe505-35a8-4590-8cd4-eaf9a1d3f0e1 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:44.624012 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-06afe505-35a8-4590-8cd4-eaf9a1d3f0e1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 14:37:44.624012 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:44.624012 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:37:44.624012 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:37:44.624012 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:37:44.624012 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:37:44.624012 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:37:44.624012 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:37:44.624012 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:37:44.624012 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:37:44.624012 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:37:44.624012 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:37:44.624012 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:37:44.624012 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:37:44.624012 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:37:44.624012 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:37:44.625226 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:37:44.625226 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:37:44.625226 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:37:44.625226 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:37:44.625226 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:37:44.625226 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:37:44.625226 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:37:44.625226 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:37:44.625226 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:37:44.625226 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:37:44.625226 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:37:44.625226 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:37:44.625226 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:37:44.625226 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:37:44.625226 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:37:44.625226 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:37:44.625226 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:37:44.625226 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:37:44.626214 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:37:44.626214 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:37:44.626214 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:37:44.626214 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:37:44.626214 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:37:44.626214 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:37:44.626214 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:37:44.626214 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:37:44.626214 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 14:37:44.626214 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:37:44.626214 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 627/1253] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:37:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:37:44.631418 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e673fceb-9d49-493d-94a9-f95bc1adce63 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:44.694122 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:44.739867 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e673fceb-9d49-493d-94a9-f95bc1adce63 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:44.782746 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196edd2-ec6d-40ee-a584-a10d322b5ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:44.783669 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196edd2-ec6d-40ee-a584-a10d322b5ff4 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:44.784030 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196edd2-ec6d-40ee-a584-a10d322b5ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:44.784434 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196edd2-ec6d-40ee-a584-a10d322b5ff4 None admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:44.785536 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2196edd2-ec6d-40ee-a584-a10d322b5ff4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:44.785934 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2196edd2-ec6d-40ee-a584-a10d322b5ff4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:44.787583 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2196edd2-ec6d-40ee-a584-a10d322b5ff4 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:44.809748 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 627/1254] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Jul 20 14:37:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:44.816848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afe505-35a8-4590-8cd4-eaf9a1d3f0e1 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:44.926609 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afe505-35a8-4590-8cd4-eaf9a1d3f0e1 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:44.963399 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087e08f-b6ff-4159-be57-d2e0bf6a99e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:44.964431 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087e08f-b6ff-4159-be57-d2e0bf6a99e0 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:44.964763 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087e08f-b6ff-4159-be57-d2e0bf6a99e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:44.965099 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087e08f-b6ff-4159-be57-d2e0bf6a99e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:45.026543 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:45.040348 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1087e08f-b6ff-4159-be57-d2e0bf6a99e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:45.042070 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1087e08f-b6ff-4159-be57-d2e0bf6a99e0 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:45.159411 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 628/1255] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:37:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 14:37:45.165461 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196edd2-ec6d-40ee-a584-a10d322b5ff4 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:45.284075 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196edd2-ec6d-40ee-a584-a10d322b5ff4 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:45.301466 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:45.331568 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe7bc5-713d-4d06-90da-7258a14b2418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:45.332470 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe7bc5-713d-4d06-90da-7258a14b2418 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:45.332731 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe7bc5-713d-4d06-90da-7258a14b2418 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:45.333049 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe7bc5-713d-4d06-90da-7258a14b2418 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:45.345937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe7bc5-713d-4d06-90da-7258a14b2418 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:45.347580 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe7bc5-713d-4d06-90da-7258a14b2418 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:45.367977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-1afe7bc5-713d-4d06-90da-7258a14b2418 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:37:45.367977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:45.367977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:37:45.367977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:37:45.367977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:37:45.367977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:37:45.367977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:37:45.367977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:37:45.367977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:37:45.367977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:37:45.367977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:37:45.367977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:37:45.367977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:37:45.367977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:37:45.367977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:37:45.369194 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:37:45.369194 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:37:45.369194 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:37:45.369194 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:37:45.369194 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:37:45.369194 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:37:45.369194 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:37:45.369194 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:37:45.369194 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:37:45.369194 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:37:45.369194 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:37:45.369194 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:37:45.369194 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:37:45.369194 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:37:45.369194 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:37:45.369194 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:37:45.369194 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:37:45.375812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:37:45.375812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:37:45.375812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:37:45.375812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:37:45.375812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 628/1256] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:37:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:37:45.376186 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087e08f-b6ff-4159-be57-d2e0bf6a99e0 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:45.484072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:45.496892 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087e08f-b6ff-4159-be57-d2e0bf6a99e0 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:45.533599 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd8ab9-20fb-4772-9f20-80cb27974b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:45.534488 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd8ab9-20fb-4772-9f20-80cb27974b23 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:45.534860 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd8ab9-20fb-4772-9f20-80cb27974b23 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:45.535207 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd8ab9-20fb-4772-9f20-80cb27974b23 None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:45.536037 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd8ab9-20fb-4772-9f20-80cb27974b23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:45.536477 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd8ab9-20fb-4772-9f20-80cb27974b23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:45.538091 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd8ab9-20fb-4772-9f20-80cb27974b23 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:45.557125 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 629/1257] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:45.563239 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe7bc5-713d-4d06-90da-7258a14b2418 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:45.688874 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe7bc5-713d-4d06-90da-7258a14b2418 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:45.723783 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67c233-b10d-4dbc-9ee7-6b946b72298e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:45.724583 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67c233-b10d-4dbc-9ee7-6b946b72298e None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:45.724863 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67c233-b10d-4dbc-9ee7-6b946b72298e None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:45.725134 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67c233-b10d-4dbc-9ee7-6b946b72298e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:45.731747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d67c233-b10d-4dbc-9ee7-6b946b72298e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:45.733207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d67c233-b10d-4dbc-9ee7-6b946b72298e None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:45.747698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-1d67c233-b10d-4dbc-9ee7-6b946b72298e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:37:45.747698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:45.747698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:37:45.747698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:37:45.747698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:37:45.747698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:37:45.747698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:37:45.747698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:37:45.747698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:37:45.747698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:37:45.747698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:37:45.747698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:37:45.747698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:37:45.747698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:37:45.747698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:37:45.748654 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:37:45.748654 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:37:45.748654 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:37:45.748654 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:37:45.748654 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:37:45.748654 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:37:45.748654 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:37:45.748654 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:37:45.748654 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:37:45.748654 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:37:45.748654 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:37:45.748654 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:37:45.748654 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:37:45.748654 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:37:45.748654 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:37:45.748654 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:37:45.748654 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:37:45.749728 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:37:45.749728 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:37:45.749728 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:37:45.749728 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:37:45.749728 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:37:45.749728 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:37:45.749728 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:37:45.749728 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:37:45.749728 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 629/1258] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:37:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:37:45.756517 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd8ab9-20fb-4772-9f20-80cb27974b23 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:45.875197 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd8ab9-20fb-4772-9f20-80cb27974b23 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:45.923059 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99db15-d785-4219-ae5c-fc8b8a8eabbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:45.923768 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99db15-d785-4219-ae5c-fc8b8a8eabbb None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:45.924054 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99db15-d785-4219-ae5c-fc8b8a8eabbb None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:45.924342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99db15-d785-4219-ae5c-fc8b8a8eabbb None admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:45.925139 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99db15-d785-4219-ae5c-fc8b8a8eabbb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:45.925450 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99db15-d785-4219-ae5c-fc8b8a8eabbb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:45.926899 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99db15-d785-4219-ae5c-fc8b8a8eabbb None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:45.941386 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 630/1259] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:37:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:45.948519 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67c233-b10d-4dbc-9ee7-6b946b72298e None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:45.977069 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:46.061871 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67c233-b10d-4dbc-9ee7-6b946b72298e None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:46.103346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735b0ca-a606-47f7-90f5-60233402a136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:46.104024 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735b0ca-a606-47f7-90f5-60233402a136 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:46.104309 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735b0ca-a606-47f7-90f5-60233402a136 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:46.104591 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735b0ca-a606-47f7-90f5-60233402a136 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:46.105437 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b735b0ca-a606-47f7-90f5-60233402a136 None admin] RBAC: Adding query filter params (role.id=49bd5abf255048a0abaa9538e309f516, scope.project.id=59333e4f593346e184877d6bd9d22026, user.id=cb857f32bbc04d82bfa681bd5659765f) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:46.105751 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b735b0ca-a606-47f7-90f5-60233402a136 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:46.107227 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b735b0ca-a606-47f7-90f5-60233402a136 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:46.113195 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 630/1260] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:37:45 2022] GET /identity/v3/role_assignments?user.id=cb857f32bbc04d82bfa681bd5659765f&scope.project.id=59333e4f593346e184877d6bd9d22026&role.id=49bd5abf255048a0abaa9538e309f516 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:48.147775 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad715b6-e169-492f-b060-e46f90589ba6 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:48.148636 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad715b6-e169-492f-b060-e46f90589ba6 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:48.148636 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad715b6-e169-492f-b060-e46f90589ba6 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:48.150863 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 631/1261] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:37:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:37:48.157328 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18e8a8-e263-4d13-9ab8-92d330f47437 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:48.157789 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18e8a8-e263-4d13-9ab8-92d330f47437 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:48.157885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18e8a8-e263-4d13-9ab8-92d330f47437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:48.184908 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-9a18e8a8-e263-4d13-9ab8-92d330f47437 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:37:48.186765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-9a18e8a8-e263-4d13-9ab8-92d330f47437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:37:48.271940 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 631/1262] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:37:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:37:48.297430 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad715b6-e169-492f-b060-e46f90589ba6 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:48.313364 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:48.418227 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad715b6-e169-492f-b060-e46f90589ba6 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:48.457085 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a01d57-cc24-45b2-9e3e-a633a0b76769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:48.457991 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a01d57-cc24-45b2-9e3e-a633a0b76769 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:48.458273 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a01d57-cc24-45b2-9e3e-a633a0b76769 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:48.458545 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a01d57-cc24-45b2-9e3e-a633a0b76769 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:48.469646 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a01d57-cc24-45b2-9e3e-a633a0b76769 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:37:48.477358 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-34a01d57-cc24-45b2-9e3e-a633a0b76769 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:37:48.477358 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:48.477358 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:37:48.477358 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:37:48.477358 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:37:48.477358 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:37:48.477358 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:37:48.477358 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:37:48.477358 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:37:48.477358 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:37:48.477358 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:37:48.477358 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:37:48.477358 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:37:48.477358 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:37:48.477358 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:37:48.477358 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:37:48.478446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:37:48.478446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:37:48.478446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:37:48.478446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:37:48.478446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:37:48.478446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:37:48.478446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:37:48.478446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:37:48.478446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:37:48.478446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:37:48.478446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:37:48.478446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:37:48.478446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:37:48.478446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:37:48.478446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:37:48.478446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:37:48.479512 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:37:48.479512 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:37:48.479512 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:37:48.479512 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:37:48.479512 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:37:48.479512 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:37:48.479512 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:37:48.479512 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:37:48.479512 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:37:48.479512 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:37:48.479512 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:37:48.479512 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:37:48.479512 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 632/1263] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:37:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:37:48.487115 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a18e8a8-e263-4d13-9ab8-92d330f47437 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:48.507233 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:48.599880 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a18e8a8-e263-4d13-9ab8-92d330f47437 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:48.639483 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae43dc-c097-46b8-8490-e5a57b26cdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:48.640159 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae43dc-c097-46b8-8490-e5a57b26cdc5 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:48.640446 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae43dc-c097-46b8-8490-e5a57b26cdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:48.640717 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae43dc-c097-46b8-8490-e5a57b26cdc5 None admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:48.641465 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ae43dc-c097-46b8-8490-e5a57b26cdc5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:48.641770 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ae43dc-c097-46b8-8490-e5a57b26cdc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:48.643252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ae43dc-c097-46b8-8490-e5a57b26cdc5 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:48.654093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 632/1264] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:37:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:48.662276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a01d57-cc24-45b2-9e3e-a633a0b76769 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:48.737327 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:48.777913 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a01d57-cc24-45b2-9e3e-a633a0b76769 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:48.817843 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac6e86-f111-46a3-952f-5e743533f63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:48.818668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac6e86-f111-46a3-952f-5e743533f63d None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:48.818955 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac6e86-f111-46a3-952f-5e743533f63d None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:48.819273 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac6e86-f111-46a3-952f-5e743533f63d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:48.831906 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ac6e86-f111-46a3-952f-5e743533f63d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:48.833404 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ac6e86-f111-46a3-952f-5e743533f63d None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:48.852322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-d9ac6e86-f111-46a3-952f-5e743533f63d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:37:48.852322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:48.852322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:37:48.852322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:37:48.852322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:37:48.852322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:37:48.852322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:37:48.852322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:37:48.852322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:37:48.852322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:37:48.852322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:37:48.852322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:37:48.852322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:37:48.852322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:37:48.852322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:37:48.853363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:37:48.853363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:37:48.853363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:37:48.853363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:37:48.853363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:37:48.853363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:37:48.853363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:37:48.853363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:37:48.853363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:37:48.853363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:37:48.853363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:37:48.853363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:37:48.853363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:37:48.853363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:37:48.853363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:37:48.853363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:37:48.853363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:37:48.854387 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:37:48.854387 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:37:48.854387 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:37:48.854387 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:37:48.854387 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 633/1265] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:37:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:37:48.860126 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae43dc-c097-46b8-8490-e5a57b26cdc5 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:48.952883 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:48.973850 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae43dc-c097-46b8-8490-e5a57b26cdc5 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:49.011174 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fab83e-f70e-449e-9d14-9f44ba834882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:49.011765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fab83e-f70e-449e-9d14-9f44ba834882 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:49.012060 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fab83e-f70e-449e-9d14-9f44ba834882 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:49.012336 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fab83e-f70e-449e-9d14-9f44ba834882 None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:49.013068 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fab83e-f70e-449e-9d14-9f44ba834882 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:49.013379 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fab83e-f70e-449e-9d14-9f44ba834882 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:49.014822 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fab83e-f70e-449e-9d14-9f44ba834882 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:49.026406 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 633/1266] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:49.032764 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac6e86-f111-46a3-952f-5e743533f63d None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:49.137650 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac6e86-f111-46a3-952f-5e743533f63d None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:49.157400 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:49.179843 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c000457-4088-41eb-bbe6-f18c1f91470a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:49.181039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c000457-4088-41eb-bbe6-f18c1f91470a None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:49.181437 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c000457-4088-41eb-bbe6-f18c1f91470a None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:49.181825 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c000457-4088-41eb-bbe6-f18c1f91470a None admin] PATH_INFO: `/v3/users/placement` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:49.191074 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c000457-4088-41eb-bbe6-f18c1f91470a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:49.192521 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c000457-4088-41eb-bbe6-f18c1f91470a None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:49.206757 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-4c000457-4088-41eb-bbe6-f18c1f91470a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 14:37:49.206757 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:49.206757 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:37:49.206757 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:37:49.206757 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:37:49.206757 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:37:49.206757 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:37:49.206757 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:37:49.206757 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:37:49.206757 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:37:49.206757 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:37:49.206757 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:37:49.206757 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:37:49.206757 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:37:49.206757 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:37:49.206757 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:37:49.207824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:37:49.207824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:37:49.207824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:37:49.207824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:37:49.207824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:37:49.207824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:37:49.207824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:37:49.207824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:37:49.207824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:37:49.207824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:37:49.207824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:37:49.207824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:37:49.207824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:37:49.207824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:37:49.207824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:37:49.207824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:37:49.207824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:37:49.207824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:37:49.208989 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:37:49.208989 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:37:49.208989 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:37:49.208989 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:37:49.208989 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:37:49.208989 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:37:49.208989 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:37:49.208989 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:37:49.208989 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 14:37:49.208989 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:37:49.208989 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 634/1267] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:37:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:37:49.216299 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fab83e-f70e-449e-9d14-9f44ba834882 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:49.330405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fab83e-f70e-449e-9d14-9f44ba834882 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:49.367209 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19600ed4-5531-44ec-936e-5172f204309a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:49.368279 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19600ed4-5531-44ec-936e-5172f204309a None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:49.368741 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19600ed4-5531-44ec-936e-5172f204309a None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:49.369198 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19600ed4-5531-44ec-936e-5172f204309a None admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:49.370172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19600ed4-5531-44ec-936e-5172f204309a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:49.370477 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19600ed4-5531-44ec-936e-5172f204309a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:49.371992 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19600ed4-5531-44ec-936e-5172f204309a None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:49.386702 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:49.399464 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 634/1268] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Jul 20 14:37:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:49.405966 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c000457-4088-41eb-bbe6-f18c1f91470a None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:49.517370 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c000457-4088-41eb-bbe6-f18c1f91470a None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:49.552963 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15cfaa6-7094-47ec-ae0f-9c69c16e8f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:49.553765 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15cfaa6-7094-47ec-ae0f-9c69c16e8f27 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:49.554046 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15cfaa6-7094-47ec-ae0f-9c69c16e8f27 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:49.554321 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15cfaa6-7094-47ec-ae0f-9c69c16e8f27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:49.565160 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15cfaa6-7094-47ec-ae0f-9c69c16e8f27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:49.566602 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15cfaa6-7094-47ec-ae0f-9c69c16e8f27 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:49.583660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-e15cfaa6-7094-47ec-ae0f-9c69c16e8f27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:37:49.583660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:49.583660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:37:49.583660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:37:49.583660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:37:49.583660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:37:49.583660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:37:49.583660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:37:49.583660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:37:49.583660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:37:49.583660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:37:49.583660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:37:49.583660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:37:49.583660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:37:49.583660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:37:49.584571 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:37:49.584571 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:37:49.584571 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:37:49.584571 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:37:49.584571 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:37:49.584571 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:37:49.584571 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:37:49.584571 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:37:49.584571 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:37:49.584571 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:37:49.584571 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:37:49.584571 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:37:49.584571 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:37:49.584571 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:37:49.584571 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:37:49.584571 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:37:49.584571 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:37:49.585606 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:37:49.585606 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:37:49.585606 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:37:49.585606 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:37:49.585606 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 635/1269] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:37:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:37:49.590881 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19600ed4-5531-44ec-936e-5172f204309a None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:49.692848 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19600ed4-5531-44ec-936e-5172f204309a None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:49.739604 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e27ab-447d-4851-9d16-5b75908b02ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:49.740271 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e27ab-447d-4851-9d16-5b75908b02ce None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:49.740555 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e27ab-447d-4851-9d16-5b75908b02ce None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:49.740830 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e27ab-447d-4851-9d16-5b75908b02ce None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:49.741927 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e27ab-447d-4851-9d16-5b75908b02ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:49.741927 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e27ab-447d-4851-9d16-5b75908b02ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:49.743238 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e27ab-447d-4851-9d16-5b75908b02ce None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:49.759593 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 635/1270] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:37:49.768004 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15cfaa6-7094-47ec-ae0f-9c69c16e8f27 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:49.803310 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:49.873287 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15cfaa6-7094-47ec-ae0f-9c69c16e8f27 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:49.908150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff5d5e-477b-4313-9d57-fb1cf90bedde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:49.908929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff5d5e-477b-4313-9d57-fb1cf90bedde None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:49.909217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff5d5e-477b-4313-9d57-fb1cf90bedde None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:49.909492 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff5d5e-477b-4313-9d57-fb1cf90bedde None admin] PATH_INFO: `/v3/projects/service` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:49.916342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff5d5e-477b-4313-9d57-fb1cf90bedde None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:49.918133 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff5d5e-477b-4313-9d57-fb1cf90bedde None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:49.932171 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-c1ff5d5e-477b-4313-9d57-fb1cf90bedde None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:37:49.932171 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:49.932171 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:37:49.932171 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:37:49.932171 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:37:49.932171 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:37:49.932171 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:37:49.932171 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:37:49.932171 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:37:49.932171 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:37:49.932171 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:37:49.932171 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:37:49.932171 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:37:49.932171 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:37:49.932171 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:37:49.933211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:37:49.933211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:37:49.933211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:37:49.933211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:37:49.933211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:37:49.933211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:37:49.933211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:37:49.933211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:37:49.933211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:37:49.933211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:37:49.933211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:37:49.933211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:37:49.933211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:37:49.933211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:37:49.933211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:37:49.933211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:37:49.933211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:37:49.934277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:37:49.934277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:37:49.934277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:37:49.934277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:37:49.934277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:37:49.934277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:37:49.934277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:37:49.934277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:37:49.934277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 636/1271] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:37:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:37:49.940030 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e27ab-447d-4851-9d16-5b75908b02ce None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:50.004370 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:50.052221 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e27ab-447d-4851-9d16-5b75908b02ce None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:50.089103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a4bd66-7241-46f1-bdfd-b6d109305681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:50.089752 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4bd66-7241-46f1-bdfd-b6d109305681 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:50.090035 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4bd66-7241-46f1-bdfd-b6d109305681 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:50.090313 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4bd66-7241-46f1-bdfd-b6d109305681 None admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:50.091170 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a4bd66-7241-46f1-bdfd-b6d109305681 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:50.091488 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a4bd66-7241-46f1-bdfd-b6d109305681 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:50.092943 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a4bd66-7241-46f1-bdfd-b6d109305681 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:50.105871 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 636/1272] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:37:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:50.112405 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff5d5e-477b-4313-9d57-fb1cf90bedde None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:50.208626 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:50.218177 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff5d5e-477b-4313-9d57-fb1cf90bedde None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:50.258518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9cb71-ba14-4d9c-96e9-62ce83e9ce79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:50.259590 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9cb71-ba14-4d9c-96e9-62ce83e9ce79 None admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:50.259896 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9cb71-ba14-4d9c-96e9-62ce83e9ce79 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:50.260210 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9cb71-ba14-4d9c-96e9-62ce83e9ce79 None admin] PATH_INFO: `/v3/projects/59333e4f593346e184877d6bd9d22026/users/cb857f32bbc04d82bfa681bd5659765f/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:50.262575 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca9cb71-ba14-4d9c-96e9-62ce83e9ce79 None admin] RBAC: Authorizing `identity:create_grant(project_id=59333e4f593346e184877d6bd9d22026, user_id=cb857f32bbc04d82bfa681bd5659765f, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:50.264501 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca9cb71-ba14-4d9c-96e9-62ce83e9ce79 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:50.280260 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 637/1273] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jul 20 14:37:50 2022] PUT /identity/v3/projects/59333e4f593346e184877d6bd9d22026/users/cb857f32bbc04d82bfa681bd5659765f/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:37:52.265822 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d0377-0c7b-45ac-87bf-05802d0626cb None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:52.266471 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d0377-0c7b-45ac-87bf-05802d0626cb None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:52.266471 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d0377-0c7b-45ac-87bf-05802d0626cb None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:52.268372 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 637/1274] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:37:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:37:52.276080 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90dbfb-795e-41be-b201-9f79f0ef997b None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:52.276522 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90dbfb-795e-41be-b201-9f79f0ef997b None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:52.276666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90dbfb-795e-41be-b201-9f79f0ef997b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:52.301365 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-ed90dbfb-795e-41be-b201-9f79f0ef997b None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:37:52.302562 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-ed90dbfb-795e-41be-b201-9f79f0ef997b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:37:52.373214 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 638/1275] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:37:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:37:52.396464 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d0377-0c7b-45ac-87bf-05802d0626cb None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:52.513987 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d0377-0c7b-45ac-87bf-05802d0626cb None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:52.532218 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:52.555688 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50d13b-e4b0-4a88-8ce8-f5d1d09c1550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:52.556490 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50d13b-e4b0-4a88-8ce8-f5d1d09c1550 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:52.556776 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50d13b-e4b0-4a88-8ce8-f5d1d09c1550 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:52.557054 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50d13b-e4b0-4a88-8ce8-f5d1d09c1550 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:52.567800 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b50d13b-e4b0-4a88-8ce8-f5d1d09c1550 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 14:37:52.575412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-6b50d13b-e4b0-4a88-8ce8-f5d1d09c1550 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:37:52.575412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:52.575412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:37:52.575412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:37:52.575412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:37:52.575412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:37:52.575412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:37:52.575412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:37:52.575412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:37:52.575412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:37:52.575412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:37:52.575412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:37:52.575412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 14:37:52.575412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 14:37:52.575412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 14:37:52.575412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 14:37:52.576482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 14:37:52.576482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 14:37:52.576482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 14:37:52.576482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 14:37:52.576482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:37:52.576482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:37:52.576482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:37:52.576482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:37:52.576482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:37:52.576482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:37:52.576482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:37:52.576482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:37:52.576482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:37:52.576482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:37:52.576482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:37:52.576482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:37:52.577523 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:37:52.577523 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:37:52.577523 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:37:52.577523 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:37:52.577523 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 14:37:52.577523 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 14:37:52.577523 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 14:37:52.577523 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 14:37:52.577523 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 14:37:52.577523 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 14:37:52.577523 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 14:37:52.577523 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:37:52.577523 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 638/1276] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jul 20 14:37:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:37:52.582923 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed90dbfb-795e-41be-b201-9f79f0ef997b None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:52.669871 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:52.690002 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed90dbfb-795e-41be-b201-9f79f0ef997b None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:52.724027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70154b3c-cccd-41b1-9424-af3b452ad24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:52.724693 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70154b3c-cccd-41b1-9424-af3b452ad24b None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:52.724981 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70154b3c-cccd-41b1-9424-af3b452ad24b None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:52.725257 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70154b3c-cccd-41b1-9424-af3b452ad24b None admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:52.726003 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70154b3c-cccd-41b1-9424-af3b452ad24b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:52.726315 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70154b3c-cccd-41b1-9424-af3b452ad24b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:52.727729 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70154b3c-cccd-41b1-9424-af3b452ad24b None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:52.735798 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 639/1277] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jul 20 14:37:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:52.741645 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50d13b-e4b0-4a88-8ce8-f5d1d09c1550 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:52.848223 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50d13b-e4b0-4a88-8ce8-f5d1d09c1550 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:52.882945 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b067ef-419e-4c19-b036-f19fa87fa79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:52.883768 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b067ef-419e-4c19-b036-f19fa87fa79a None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:52.884023 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b067ef-419e-4c19-b036-f19fa87fa79a None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:52.884331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b067ef-419e-4c19-b036-f19fa87fa79a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:52.895243 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b067ef-419e-4c19-b036-f19fa87fa79a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:52.896677 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b067ef-419e-4c19-b036-f19fa87fa79a None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:52.914347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-40b067ef-419e-4c19-b036-f19fa87fa79a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:37:52.914347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:52.914347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:37:52.914347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:37:52.914347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:37:52.914347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:37:52.914347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:37:52.914347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:37:52.914347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:37:52.914347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:37:52.914347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:37:52.914347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:37:52.914347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:37:52.914347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:37:52.914347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:37:52.915433 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:37:52.915433 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:37:52.915433 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:37:52.915433 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:37:52.915433 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:37:52.915433 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:37:52.915433 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:37:52.915433 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:37:52.915433 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:37:52.915433 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:37:52.915433 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:37:52.915433 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:37:52.915433 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:37:52.915433 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:37:52.915433 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:37:52.915433 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:37:52.915433 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:37:52.916721 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:37:52.916721 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:37:52.916721 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:37:52.916721 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:37:52.916721 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 639/1278] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:37:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:37:52.921617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70154b3c-cccd-41b1-9424-af3b452ad24b None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:53.024112 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70154b3c-cccd-41b1-9424-af3b452ad24b None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:53.061489 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598467a-645b-4507-aab4-4b261fb3550f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:53.062139 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598467a-645b-4507-aab4-4b261fb3550f None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:53.062424 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598467a-645b-4507-aab4-4b261fb3550f None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:53.062700 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598467a-645b-4507-aab4-4b261fb3550f None admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:53.063447 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5598467a-645b-4507-aab4-4b261fb3550f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:53.063752 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5598467a-645b-4507-aab4-4b261fb3550f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:53.065119 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5598467a-645b-4507-aab4-4b261fb3550f None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:53.076835 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 640/1279] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:53.082645 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b067ef-419e-4c19-b036-f19fa87fa79a None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:53.118678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:53.195619 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b067ef-419e-4c19-b036-f19fa87fa79a None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:53.230713 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b32cc-1f77-4dc7-843d-1c765d0d2dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:53.231570 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b32cc-1f77-4dc7-843d-1c765d0d2dd9 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:53.231772 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b32cc-1f77-4dc7-843d-1c765d0d2dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:53.232074 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b32cc-1f77-4dc7-843d-1c765d0d2dd9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:53.239656 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b32cc-1f77-4dc7-843d-1c765d0d2dd9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:53.241094 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b32cc-1f77-4dc7-843d-1c765d0d2dd9 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:53.256062 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-c12b32cc-1f77-4dc7-843d-1c765d0d2dd9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 14:37:53.256062 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:53.256062 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:37:53.256062 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:37:53.256062 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:37:53.256062 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:37:53.256062 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:37:53.256062 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:37:53.256062 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:37:53.256062 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:37:53.256062 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:37:53.256062 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:37:53.256062 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 14:37:53.256062 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 14:37:53.256062 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 14:37:53.256062 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 14:37:53.257905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:37:53.257905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:37:53.257905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 14:37:53.257905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:37:53.257905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 14:37:53.257905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 14:37:53.257905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:37:53.257905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:37:53.257905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:37:53.257905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:37:53.257905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:37:53.257905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:37:53.257905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:37:53.257905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:37:53.257905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:37:53.257905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:37:53.257905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:37:53.257905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:37:53.259245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:37:53.259245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:37:53.259245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 14:37:53.259245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 14:37:53.259245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 14:37:53.259245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 14:37:53.259245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 14:37:53.259245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 14:37:53.259245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 14:37:53.259245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:37:53.259245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 640/1280] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:37:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:37:53.263338 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598467a-645b-4507-aab4-4b261fb3550f None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:53.267616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:53.367547 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598467a-645b-4507-aab4-4b261fb3550f None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:53.402542 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f56117-e91a-4bb7-9bac-afa160ba99f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:53.403170 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f56117-e91a-4bb7-9bac-afa160ba99f5 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:53.403457 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f56117-e91a-4bb7-9bac-afa160ba99f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:53.403736 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f56117-e91a-4bb7-9bac-afa160ba99f5 None admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:53.404576 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f56117-e91a-4bb7-9bac-afa160ba99f5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:53.404959 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f56117-e91a-4bb7-9bac-afa160ba99f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:53.406393 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f56117-e91a-4bb7-9bac-afa160ba99f5 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:53.423701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 641/1281] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Jul 20 14:37:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:53.432286 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b32cc-1f77-4dc7-843d-1c765d0d2dd9 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:53.544906 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:53.557330 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b32cc-1f77-4dc7-843d-1c765d0d2dd9 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:53.596481 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256b5aa-1c9b-4da1-80b0-51d592b64fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:53.597134 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256b5aa-1c9b-4da1-80b0-51d592b64fbf None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:53.597450 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256b5aa-1c9b-4da1-80b0-51d592b64fbf None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:53.597725 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256b5aa-1c9b-4da1-80b0-51d592b64fbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:53.679870 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256b5aa-1c9b-4da1-80b0-51d592b64fbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:53.681283 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256b5aa-1c9b-4da1-80b0-51d592b64fbf None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:53.776191 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:53.786691 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 641/1282] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jul 20 14:37:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 14:37:53.796165 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f56117-e91a-4bb7-9bac-afa160ba99f5 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:53.860379 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:53.900858 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f56117-e91a-4bb7-9bac-afa160ba99f5 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:53.936266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715260dd-c1ed-4282-b158-2d6850848226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:53.936992 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715260dd-c1ed-4282-b158-2d6850848226 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:53.937273 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715260dd-c1ed-4282-b158-2d6850848226 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:53.937544 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715260dd-c1ed-4282-b158-2d6850848226 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:53.947942 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715260dd-c1ed-4282-b158-2d6850848226 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:53.949402 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715260dd-c1ed-4282-b158-2d6850848226 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:53.967071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-715260dd-c1ed-4282-b158-2d6850848226 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:37:53.967071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:53.967071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:37:53.967071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:37:53.967071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:37:53.967071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:37:53.967071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:37:53.967071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:37:53.967071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:37:53.967071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:37:53.967071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:37:53.967071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:37:53.967071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 14:37:53.967071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 14:37:53.967071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 14:37:53.968067 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 14:37:53.968067 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:37:53.968067 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:37:53.968067 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:37:53.968067 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:37:53.968067 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:37:53.968067 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:37:53.968067 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:37:53.968067 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:37:53.968067 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:37:53.968067 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:37:53.968067 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:37:53.968067 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:37:53.968067 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:37:53.968067 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:37:53.968067 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:37:53.968067 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:37:53.968982 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 14:37:53.968982 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 14:37:53.968982 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 14:37:53.968982 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:37:53.968982 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 642/1283] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:37:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:37:53.977483 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256b5aa-1c9b-4da1-80b0-51d592b64fbf None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:54.080734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256b5aa-1c9b-4da1-80b0-51d592b64fbf None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:54.117153 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb0005-ce57-4d4b-b505-13825763b86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:54.118113 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb0005-ce57-4d4b-b505-13825763b86d None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:54.118494 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb0005-ce57-4d4b-b505-13825763b86d None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:54.118864 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb0005-ce57-4d4b-b505-13825763b86d None admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:54.119990 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fb0005-ce57-4d4b-b505-13825763b86d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:54.120412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fb0005-ce57-4d4b-b505-13825763b86d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:54.122568 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fb0005-ce57-4d4b-b505-13825763b86d None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:54.135075 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 642/1284] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jul 20 14:37:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:54.140903 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715260dd-c1ed-4282-b158-2d6850848226 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:54.251324 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715260dd-c1ed-4282-b158-2d6850848226 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:54.265809 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:54.288267 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3622a-bf12-4cef-8970-b4dabde8ea08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:54.289066 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3622a-bf12-4cef-8970-b4dabde8ea08 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:54.289350 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3622a-bf12-4cef-8970-b4dabde8ea08 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:54.289628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3622a-bf12-4cef-8970-b4dabde8ea08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:54.296906 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d3622a-bf12-4cef-8970-b4dabde8ea08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:54.298355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d3622a-bf12-4cef-8970-b4dabde8ea08 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:54.312211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-e0d3622a-bf12-4cef-8970-b4dabde8ea08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:37:54.312211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:54.312211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:37:54.312211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:37:54.312211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:37:54.312211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:37:54.312211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:37:54.312211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:37:54.312211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:37:54.312211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:37:54.312211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:37:54.312211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:37:54.312211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 14:37:54.312211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 14:37:54.312211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 14:37:54.313189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 14:37:54.313189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:37:54.313189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:37:54.313189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:37:54.313189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:37:54.313189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:37:54.313189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:37:54.313189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:37:54.313189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 14:37:54.313189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:37:54.313189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:37:54.313189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:37:54.313189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:37:54.313189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:37:54.313189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:37:54.313189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:37:54.313189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:37:54.314198 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 14:37:54.314198 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 14:37:54.314198 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 14:37:54.314198 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 14:37:54.314198 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 14:37:54.314198 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 14:37:54.314198 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 14:37:54.314198 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 14:37:54.314198 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 643/1285] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jul 20 14:37:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:37:54.321589 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb0005-ce57-4d4b-b505-13825763b86d None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:54.374680 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:54.431070 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb0005-ce57-4d4b-b505-13825763b86d None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:54.466821 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cdeb50-71da-4fcd-8a7b-f21dfeaa8d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:54.467464 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cdeb50-71da-4fcd-8a7b-f21dfeaa8d44 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:54.467781 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cdeb50-71da-4fcd-8a7b-f21dfeaa8d44 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:54.468061 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cdeb50-71da-4fcd-8a7b-f21dfeaa8d44 None admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:54.468970 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cdeb50-71da-4fcd-8a7b-f21dfeaa8d44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:54.469279 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cdeb50-71da-4fcd-8a7b-f21dfeaa8d44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:54.470732 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cdeb50-71da-4fcd-8a7b-f21dfeaa8d44 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:54.483974 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 643/1286] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jul 20 14:37:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:54.490348 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3622a-bf12-4cef-8970-b4dabde8ea08 None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:54.595204 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3622a-bf12-4cef-8970-b4dabde8ea08 None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:54.630262 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71df5d2-5c67-4c5d-8850-fad97cb1f5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:54.630951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71df5d2-5c67-4c5d-8850-fad97cb1f5a6 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:54.631270 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71df5d2-5c67-4c5d-8850-fad97cb1f5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:54.631549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71df5d2-5c67-4c5d-8850-fad97cb1f5a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:54.632386 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71df5d2-5c67-4c5d-8850-fad97cb1f5a6 None admin] RBAC: Adding query filter params (role.id=49bd5abf255048a0abaa9538e309f516, scope.project.id=59333e4f593346e184877d6bd9d22026, user.id=cb857f32bbc04d82bfa681bd5659765f) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:54.632708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71df5d2-5c67-4c5d-8850-fad97cb1f5a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:54.634146 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71df5d2-5c67-4c5d-8850-fad97cb1f5a6 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:54.640164 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 644/1287] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jul 20 14:37:54 2022] GET /identity/v3/role_assignments?user.id=cb857f32bbc04d82bfa681bd5659765f&scope.project.id=59333e4f593346e184877d6bd9d22026&role.id=49bd5abf255048a0abaa9538e309f516 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:37:56.686426 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0b307-e059-44cb-b955-5d3aba73fe6c None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:56.687382 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0b307-e059-44cb-b955-5d3aba73fe6c None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:56.687382 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0b307-e059-44cb-b955-5d3aba73fe6c None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:56.689570 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 644/1288] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:37:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:37:56.696027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25d414-7672-4c90-8aac-f79cbc62143a None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:56.696710 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25d414-7672-4c90-8aac-f79cbc62143a None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:56.696907 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25d414-7672-4c90-8aac-f79cbc62143a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:56.716998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-4d25d414-7672-4c90-8aac-f79cbc62143a None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:37:56.718193 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-4d25d414-7672-4c90-8aac-f79cbc62143a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:37:56.748920 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:56.794797 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 645/1289] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:37:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:37:56.830312 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0b307-e059-44cb-b955-5d3aba73fe6c None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:56.946686 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0b307-e059-44cb-b955-5d3aba73fe6c None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:56.957317 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:56.986833 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f6a05-f154-4da7-ab04-67ebb6f4478c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:56.987679 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f6a05-f154-4da7-ab04-67ebb6f4478c None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:56.987975 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f6a05-f154-4da7-ab04-67ebb6f4478c None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:56.988244 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f6a05-f154-4da7-ab04-67ebb6f4478c None admin] PATH_INFO: `/v3/services/placement` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:56.988904 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f6a05-f154-4da7-ab04-67ebb6f4478c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:56.990301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f6a05-f154-4da7-ab04-67ebb6f4478c None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:57.002409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-4c0f6a05-f154-4da7-ab04-67ebb6f4478c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 14:37:57.002409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:57.002409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 14:37:57.002409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 14:37:57.002409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 14:37:57.002409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 14:37:57.002409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 14:37:57.002409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:37:57.002409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 14:37:57.002409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:37:57.002409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 14:37:57.002409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:37:57.002409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:37:57.002409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:37:57.002409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:37:57.002409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:37:57.002409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:37:57.003614 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:37:57.003614 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:37:57.003614 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:37:57.003614 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:37:57.003614 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:37:57.003614 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:37:57.003614 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 14:37:57.003614 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 14:37:57.003614 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 14:37:57.003614 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 14:37:57.003614 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:37:57.003614 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:37:57.003614 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:37:57.003614 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:37:57.003614 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:37:57.003614 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:37:57.004584 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:37:57.004584 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:37:57.004584 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:37:57.004584 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:37:57.004584 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:37:57.004584 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:37:57.004584 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:37:57.004584 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:37:57.004584 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:37:57.004584 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:37:57.004584 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 14:37:57.004584 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:37:57.004584 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 14:37:57.004584 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:37:57.004584 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 645/1290] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Jul 20 14:37:56 2022] GET /identity/v3/services/placement => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:37:57.009731 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25d414-7672-4c90-8aac-f79cbc62143a None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:57.113596 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25d414-7672-4c90-8aac-f79cbc62143a None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:57.148732 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae67c93d-e067-47f4-9d81-5773a64f4a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:57.149389 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67c93d-e067-47f4-9d81-5773a64f4a92 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:57.149672 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67c93d-e067-47f4-9d81-5773a64f4a92 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:57.149946 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67c93d-e067-47f4-9d81-5773a64f4a92 None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:57.150682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae67c93d-e067-47f4-9d81-5773a64f4a92 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:57.150991 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae67c93d-e067-47f4-9d81-5773a64f4a92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:57.152467 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae67c93d-e067-47f4-9d81-5773a64f4a92 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:57.157579 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 646/1291] 10.222.0.15 () {56 vars in 1116 bytes} [Wed Jul 20 14:37:57 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:57.164013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f6a05-f154-4da7-ab04-67ebb6f4478c None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:57.285249 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f6a05-f154-4da7-ab04-67ebb6f4478c None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:57.320692 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598f8fa-15b7-4887-b68d-68ecf942eafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:57.321357 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598f8fa-15b7-4887-b68d-68ecf942eafd None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:57.321650 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598f8fa-15b7-4887-b68d-68ecf942eafd None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:57.321926 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598f8fa-15b7-4887-b68d-68ecf942eafd None admin] PATH_INFO: `/v3/services` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:57.322693 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598f8fa-15b7-4887-b68d-68ecf942eafd None admin] RBAC: Adding query filter params (type=placement) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:37:57.322959 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598f8fa-15b7-4887-b68d-68ecf942eafd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:57.324387 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598f8fa-15b7-4887-b68d-68ecf942eafd None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:57.329037 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 646/1292] 10.222.0.15 () {56 vars in 1116 bytes} [Wed Jul 20 14:37:57 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:37:59.324775 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0cda2-084f-444a-aa0a-10be94965f6b None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:59.325696 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0cda2-084f-444a-aa0a-10be94965f6b None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:59.325696 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0cda2-084f-444a-aa0a-10be94965f6b None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:59.327846 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 647/1293] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:37:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:37:59.336515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c9950-5afe-48de-a96d-cb8277515c64 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:59.336942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c9950-5afe-48de-a96d-cb8277515c64 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:59.337099 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c9950-5afe-48de-a96d-cb8277515c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:59.357942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-184c9950-5afe-48de-a96d-cb8277515c64 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:37:59.359218 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-184c9950-5afe-48de-a96d-cb8277515c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:37:59.391739 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:59.433164 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 647/1294] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:37:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:37:59.456155 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf0cda2-084f-444a-aa0a-10be94965f6b None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:37:59.496531 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:37:59.570843 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf0cda2-084f-444a-aa0a-10be94965f6b None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:37:59.606013 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba08135-5642-4c5f-8c18-c72ed7566557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:37:59.606611 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba08135-5642-4c5f-8c18-c72ed7566557 None admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:37:59.606893 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba08135-5642-4c5f-8c18-c72ed7566557 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:37:59.607189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba08135-5642-4c5f-8c18-c72ed7566557 None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:37:59.607910 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba08135-5642-4c5f-8c18-c72ed7566557 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:37:59.609324 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba08135-5642-4c5f-8c18-c72ed7566557 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:37:59.627029 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 648/1295] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Jul 20 14:37:59 2022] POST /identity/v3/services => generated 246 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:38:01.630977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4bf4d-599d-4988-a8d1-d2b801b6fc3b None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:38:01.631579 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4bf4d-599d-4988-a8d1-d2b801b6fc3b None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:38:01.631579 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4bf4d-599d-4988-a8d1-d2b801b6fc3b None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:38:01.632964 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 648/1296] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:38:01.639416 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b4c1f-5741-439b-9983-3ea5af3c2509 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:38:01.639842 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b4c1f-5741-439b-9983-3ea5af3c2509 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:38:01.639993 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b4c1f-5741-439b-9983-3ea5af3c2509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:38:01.664419 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-b89b4c1f-5741-439b-9983-3ea5af3c2509 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:38:01.665622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-b89b4c1f-5741-439b-9983-3ea5af3c2509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:38:01.751912 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 649/1297] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:38:01 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:38:01.774449 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4bf4d-599d-4988-a8d1-d2b801b6fc3b None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:38:01.892892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4bf4d-599d-4988-a8d1-d2b801b6fc3b None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:38:01.931998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8826b802-114c-4c6b-980c-3ba890bcc4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:38:01.932800 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826b802-114c-4c6b-980c-3ba890bcc4d0 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:38:01.933086 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826b802-114c-4c6b-980c-3ba890bcc4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:38:01.933356 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826b802-114c-4c6b-980c-3ba890bcc4d0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:38:01.934024 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8826b802-114c-4c6b-980c-3ba890bcc4d0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:38:01.935471 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8826b802-114c-4c6b-980c-3ba890bcc4d0 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:38:01.947276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-8826b802-114c-4c6b-980c-3ba890bcc4d0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 14:38:01.947276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:38:01.947276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 14:38:01.947276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 14:38:01.947276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 14:38:01.947276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 14:38:01.947276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 14:38:01.947276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:38:01.947276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 14:38:01.947276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:38:01.947276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 14:38:01.947276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:38:01.947276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:38:01.947276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:38:01.947276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:38:01.947276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:38:01.947276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:38:01.948482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:38:01.948482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:38:01.948482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:38:01.948482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:38:01.948482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:38:01.948482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:38:01.948482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 14:38:01.948482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 14:38:01.948482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 14:38:01.948482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 14:38:01.948482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:38:01.948482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:38:01.948482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:38:01.948482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:38:01.948482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:38:01.948482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:38:01.949353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:38:01.949353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:38:01.949353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:38:01.949353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:38:01.949353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:38:01.949353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:38:01.949353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:38:01.949353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:38:01.949353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:38:01.949353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:38:01.949353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 14:38:01.949353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:38:01.949353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 14:38:01.949353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:38:01.949353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 649/1298] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Jul 20 14:38:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:38:01.955297 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b4c1f-5741-439b-9983-3ea5af3c2509 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:38:01.974280 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:38:02.056634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b4c1f-5741-439b-9983-3ea5af3c2509 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:38:02.091377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24103812-14f3-4213-9f2b-a5527541c78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:38:02.092040 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24103812-14f3-4213-9f2b-a5527541c78d None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:38:02.092324 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24103812-14f3-4213-9f2b-a5527541c78d None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:38:02.092602 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24103812-14f3-4213-9f2b-a5527541c78d None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:38:02.093333 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24103812-14f3-4213-9f2b-a5527541c78d None admin] RBAC: Adding query filter params (name=placement) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:38:02.093642 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24103812-14f3-4213-9f2b-a5527541c78d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:38:02.095093 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24103812-14f3-4213-9f2b-a5527541c78d None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:38:02.100226 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 650/1299] 10.222.0.15 () {56 vars in 1116 bytes} [Wed Jul 20 14:38:01 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:38:02.109431 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8826b802-114c-4c6b-980c-3ba890bcc4d0 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:38:02.142287 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:38:02.212461 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8826b802-114c-4c6b-980c-3ba890bcc4d0 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:38:02.253071 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccca8c7-02b1-40c5-b2d6-6409137d9e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:38:02.253753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccca8c7-02b1-40c5-b2d6-6409137d9e5d None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:38:02.254039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccca8c7-02b1-40c5-b2d6-6409137d9e5d None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:38:02.254318 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccca8c7-02b1-40c5-b2d6-6409137d9e5d None admin] PATH_INFO: `/v3/endpoints` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:38:02.255148 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccca8c7-02b1-40c5-b2d6-6409137d9e5d None admin] RBAC: Adding query filter params (interface=public, service_id=300c0e51c2de4d919f6c4fb8516c4839, region_id=RegionOne) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:38:02.255464 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccca8c7-02b1-40c5-b2d6-6409137d9e5d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:38:02.256855 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccca8c7-02b1-40c5-b2d6-6409137d9e5d None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:38:02.262959 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 650/1300] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Jul 20 14:38:02 2022] GET /identity/v3/endpoints?service_id=300c0e51c2de4d919f6c4fb8516c4839&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:38:02.269442 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24103812-14f3-4213-9f2b-a5527541c78d None admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:38:02.372701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:38:02.391694 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24103812-14f3-4213-9f2b-a5527541c78d None admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:38:02.430099 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c5ed60-a1c3-42fa-8cee-e6246234fd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:38:02.431159 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5ed60-a1c3-42fa-8cee-e6246234fd8d None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:38:02.431587 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5ed60-a1c3-42fa-8cee-e6246234fd8d None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:38:02.432021 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5ed60-a1c3-42fa-8cee-e6246234fd8d None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:38:02.433155 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c5ed60-a1c3-42fa-8cee-e6246234fd8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:38:02.435359 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c5ed60-a1c3-42fa-8cee-e6246234fd8d None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:38:02.443395 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 651/1301] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Jul 20 14:38:02 2022] GET /identity/v3/services => generated 2191 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:38:04.446791 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400b174-d7a1-4060-9516-8e6e9ee3ef47 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:38:04.447380 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400b174-d7a1-4060-9516-8e6e9ee3ef47 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:38:04.447380 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400b174-d7a1-4060-9516-8e6e9ee3ef47 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:38:04.448794 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 651/1302] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:38:04.455473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5e61d-240e-4ad9-84db-3192eed60c8e None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:38:04.456156 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5e61d-240e-4ad9-84db-3192eed60c8e None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:38:04.456350 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5e61d-240e-4ad9-84db-3192eed60c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:38:04.484110 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-96e5e61d-240e-4ad9-84db-3192eed60c8e None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:38:04.485312 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-96e5e61d-240e-4ad9-84db-3192eed60c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:38:04.555472 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 652/1303] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:38:04 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 14:38:04.580630 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400b174-d7a1-4060-9516-8e6e9ee3ef47 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:38:04.685811 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:38:04.694033 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400b174-d7a1-4060-9516-8e6e9ee3ef47 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:38:04.732927 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015654ee-7a26-419b-a6b7-aa61b18e0966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:38:04.733688 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015654ee-7a26-419b-a6b7-aa61b18e0966 None admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:38:04.733978 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015654ee-7a26-419b-a6b7-aa61b18e0966 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:38:04.734254 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015654ee-7a26-419b-a6b7-aa61b18e0966 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:38:04.734914 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015654ee-7a26-419b-a6b7-aa61b18e0966 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:38:04.736326 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015654ee-7a26-419b-a6b7-aa61b18e0966 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:38:04.749108 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application [None req-015654ee-7a26-419b-a6b7-aa61b18e0966 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 14:38:04.749108 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:38:04.749108 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 14:38:04.749108 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 14:38:04.749108 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 14:38:04.749108 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 14:38:04.749108 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 14:38:04.749108 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:38:04.749108 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 14:38:04.749108 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:38:04.749108 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 14:38:04.749108 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:38:04.749108 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 14:38:04.749108 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 14:38:04.749108 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 14:38:04.749108 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 14:38:04.749108 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 14:38:04.750358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 14:38:04.750358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 14:38:04.750358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 14:38:04.750358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 14:38:04.750358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 14:38:04.750358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 14:38:04.750358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 14:38:04.750358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 14:38:04.750358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 14:38:04.750358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 14:38:04.750358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 14:38:04.750358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 14:38:04.750358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 14:38:04.750358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 14:38:04.750358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 14:38:04.750358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 14:38:04.751570 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 14:38:04.751570 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application with Lock( Jul 20 14:38:04.751570 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 14:38:04.751570 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self._enter() Jul 20 14:38:04.751570 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 14:38:04.751570 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 14:38:04.751570 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 14:38:04.751570 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application return self.creator() Jul 20 14:38:04.751570 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 14:38:04.751570 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application created_value = creator( Jul 20 14:38:04.751570 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 14:38:04.751570 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 14:38:04.751570 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 14:38:04.751570 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: ERROR keystone.server.flask.application Jul 20 14:38:04.751570 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 652/1304] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Jul 20 14:38:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 14:38:04.757466 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e5e61d-240e-4ad9-84db-3192eed60c8e None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:38:04.847546 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:38:04.882307 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e5e61d-240e-4ad9-84db-3192eed60c8e None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:38:04.919625 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d3669-14ad-42c3-8c48-24ccd74b8844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:38:04.920309 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d3669-14ad-42c3-8c48-24ccd74b8844 None admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:38:04.920593 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d3669-14ad-42c3-8c48-24ccd74b8844 None admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:38:04.920868 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d3669-14ad-42c3-8c48-24ccd74b8844 None admin] PATH_INFO: `/v3/services` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:38:04.921635 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568d3669-14ad-42c3-8c48-24ccd74b8844 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:38:04.921951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568d3669-14ad-42c3-8c48-24ccd74b8844 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:38:04.923399 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568d3669-14ad-42c3-8c48-24ccd74b8844 None admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:38:04.928437 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 653/1305] 10.222.0.15 () {56 vars in 1116 bytes} [Wed Jul 20 14:38:04 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:38:04.934391 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015654ee-7a26-419b-a6b7-aa61b18e0966 None admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:38:05.050470 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015654ee-7a26-419b-a6b7-aa61b18e0966 None admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:38:05.085786 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b9395-45a3-4408-b3a0-614bedc388a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:38:05.086443 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b9395-45a3-4408-b3a0-614bedc388a5 None admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:38:05.086727 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b9395-45a3-4408-b3a0-614bedc388a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:38:05.086999 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b9395-45a3-4408-b3a0-614bedc388a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:38:05.087775 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9b9395-45a3-4408-b3a0-614bedc388a5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:38:05.089160 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9b9395-45a3-4408-b3a0-614bedc388a5 None admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:38:05.113318 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 653/1306] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Jul 20 14:38:04 2022] POST /identity/v3/endpoints => generated 331 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:38:14.925400 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=30 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:38:15.108503 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:39:08.000606 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08e35d-3a51-42b1-9c2a-fc4c5746a71b None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:08.001640 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08e35d-3a51-42b1-9c2a-fc4c5746a71b None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:08.002129 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08e35d-3a51-42b1-9c2a-fc4c5746a71b None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:08.007815 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 654/1307] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:39:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:39:08.009224 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dfc70c-49a3-49f3-9afe-3a12e662f76b None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:08.009653 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dfc70c-49a3-49f3-9afe-3a12e662f76b None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:08.009855 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dfc70c-49a3-49f3-9afe-3a12e662f76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:08.097611 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-c3dfc70c-49a3-49f3-9afe-3a12e662f76b None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:39:08.098896 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-c3dfc70c-49a3-49f3-9afe-3a12e662f76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:08.341996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 654/1308] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:39:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:39:08.401068 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb582d-b3e7-4036-8bc5-780169dc2198 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:08.401503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb582d-b3e7-4036-8bc5-780169dc2198 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:08.401621 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb582d-b3e7-4036-8bc5-780169dc2198 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:08.403449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 655/1309] 10.222.0.15 () {54 vars in 965 bytes} [Wed Jul 20 14:39:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:39:08.411826 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3981e4-9917-45b9-a2e7-ba70a806a3e1 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:08.412406 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3981e4-9917-45b9-a2e7-ba70a806a3e1 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:08.412900 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3981e4-9917-45b9-a2e7-ba70a806a3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:08.499037 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-ab3981e4-9917-45b9-a2e7-ba70a806a3e1 None None] MFA Rules not processed for user `7658ffde740d446682c1876662815987`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:39:08.500416 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-ab3981e4-9917-45b9-a2e7-ba70a806a3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:08.522758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:39:08.591226 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 655/1310] 10.222.0.15 () {58 vars in 1062 bytes} [Wed Jul 20 14:39:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:39:08.599492 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb582d-b3e7-4036-8bc5-780169dc2198 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:39:08.601447 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-d4eb582d-b3e7-4036-8bc5-780169dc2198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:08.717675 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb582d-b3e7-4036-8bc5-780169dc2198 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:39:08.797412 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056ee19-fff0-4d7c-9e6c-d2964b0d5807 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:39:08.798348 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056ee19-fff0-4d7c-9e6c-d2964b0d5807 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:08.798738 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056ee19-fff0-4d7c-9e6c-d2964b0d5807 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:08.799168 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056ee19-fff0-4d7c-9e6c-d2964b0d5807 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:08.800942 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-4056ee19-fff0-4d7c-9e6c-d2964b0d5807 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:08.862386 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:39:08.923203 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4056ee19-fff0-4d7c-9e6c-d2964b0d5807 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:39:08.924815 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4056ee19-fff0-4d7c-9e6c-d2964b0d5807 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:39:08.945084 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 656/1311] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:39:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:39:11.230991 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fdffe-b006-4c2c-9df8-53e5d4ec5948 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:11.231627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fdffe-b006-4c2c-9df8-53e5d4ec5948 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:11.231627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fdffe-b006-4c2c-9df8-53e5d4ec5948 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:11.233017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 656/1312] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:39:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:39:11.238352 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf09be-b9fb-4145-8895-1084ee1bcd67 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:11.238810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf09be-b9fb-4145-8895-1084ee1bcd67 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:11.239028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf09be-b9fb-4145-8895-1084ee1bcd67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:11.319936 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-7dbf09be-b9fb-4145-8895-1084ee1bcd67 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:39:11.321179 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-7dbf09be-b9fb-4145-8895-1084ee1bcd67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:11.423846 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 657/1313] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:39:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:39:11.461403 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620fdffe-b006-4c2c-9df8-53e5d4ec5948 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:39:11.463036 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-620fdffe-b006-4c2c-9df8-53e5d4ec5948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:11.578873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620fdffe-b006-4c2c-9df8-53e5d4ec5948 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:39:11.658773 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:39:11.661704 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5346157-bcf3-4186-811d-a10167062621 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:39:11.662376 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5346157-bcf3-4186-811d-a10167062621 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:11.662647 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5346157-bcf3-4186-811d-a10167062621 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:11.662934 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5346157-bcf3-4186-811d-a10167062621 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:11.665965 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-a5346157-bcf3-4186-811d-a10167062621 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:11.773869 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5346157-bcf3-4186-811d-a10167062621 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:39:11.775294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5346157-bcf3-4186-811d-a10167062621 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:39:11.794306 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 657/1314] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:39:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:39:14.108044 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17685ed9-9ecf-4a5e-8401-7f381c9440b9 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:14.108605 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17685ed9-9ecf-4a5e-8401-7f381c9440b9 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:14.108605 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17685ed9-9ecf-4a5e-8401-7f381c9440b9 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:14.109967 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 658/1315] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:39:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:39:14.115336 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226e307-0c55-4cf4-b1a7-aa2057e8eda7 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:14.115759 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226e307-0c55-4cf4-b1a7-aa2057e8eda7 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:14.116050 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226e307-0c55-4cf4-b1a7-aa2057e8eda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:14.183356 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-a226e307-0c55-4cf4-b1a7-aa2057e8eda7 None None] MFA Rules not processed for user `55e7a5b2a35b46a0a3eedafbe6c53a2c`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:39:14.184549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-a226e307-0c55-4cf4-b1a7-aa2057e8eda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:14.241321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:39:14.286933 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 658/1316] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:39:14 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:39:14.337675 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac00ef9-08f2-4790-a57a-7bf7ce205b92 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:14.337961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac00ef9-08f2-4790-a57a-7bf7ce205b92 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:14.338238 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac00ef9-08f2-4790-a57a-7bf7ce205b92 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:14.339673 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 659/1317] 10.222.0.15 () {54 vars in 965 bytes} [Wed Jul 20 14:39:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:39:14.346217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b5a84-13ff-4aac-a3be-017b5b8654b2 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:14.346593 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b5a84-13ff-4aac-a3be-017b5b8654b2 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:14.346975 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b5a84-13ff-4aac-a3be-017b5b8654b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:14.426321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-538b5a84-13ff-4aac-a3be-017b5b8654b2 None None] MFA Rules not processed for user `7658ffde740d446682c1876662815987`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:39:14.427597 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-538b5a84-13ff-4aac-a3be-017b5b8654b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:14.507226 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 659/1318] 10.222.0.15 () {58 vars in 1062 bytes} [Wed Jul 20 14:39:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:39:14.515699 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac00ef9-08f2-4790-a57a-7bf7ce205b92 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:39:14.517275 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-aac00ef9-08f2-4790-a57a-7bf7ce205b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:14.527075 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:39:14.632613 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac00ef9-08f2-4790-a57a-7bf7ce205b92 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:39:14.717074 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14c77c-cb22-4cfb-b218-a06b41c5ed22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:39:14.717645 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14c77c-cb22-4cfb-b218-a06b41c5ed22 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:14.717927 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14c77c-cb22-4cfb-b218-a06b41c5ed22 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:14.718194 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14c77c-cb22-4cfb-b218-a06b41c5ed22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:14.719870 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-cb14c77c-cb22-4cfb-b218-a06b41c5ed22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:14.783603 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:39:14.830513 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb14c77c-cb22-4cfb-b218-a06b41c5ed22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:39:14.831986 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb14c77c-cb22-4cfb-b218-a06b41c5ed22 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:39:14.850602 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 660/1319] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:39:14 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:39:17.773482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3e7aa-f217-4bbf-b90a-9c798f043e98 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:17.774284 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3e7aa-f217-4bbf-b90a-9c798f043e98 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:17.774284 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3e7aa-f217-4bbf-b90a-9c798f043e98 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:17.776479 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 660/1320] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:39:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:39:17.785309 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4412990-37d1-40ec-a8c4-ba8ba57258aa None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:17.785744 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4412990-37d1-40ec-a8c4-ba8ba57258aa None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:17.785931 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4412990-37d1-40ec-a8c4-ba8ba57258aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:17.858955 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-c4412990-37d1-40ec-a8c4-ba8ba57258aa None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:39:17.860195 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-c4412990-37d1-40ec-a8c4-ba8ba57258aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:17.966007 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 661/1321] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:39:17 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:39:18.005643 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a3e7aa-f217-4bbf-b90a-9c798f043e98 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:39:18.007223 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-d4a3e7aa-f217-4bbf-b90a-9c798f043e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:18.058765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:39:18.116649 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a3e7aa-f217-4bbf-b90a-9c798f043e98 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:39:18.202208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad174f-1f2c-4f55-8536-20be2a2388c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:39:18.202861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad174f-1f2c-4f55-8536-20be2a2388c5 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:18.203175 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad174f-1f2c-4f55-8536-20be2a2388c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:18.203450 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad174f-1f2c-4f55-8536-20be2a2388c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:18.205086 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-f3ad174f-1f2c-4f55-8536-20be2a2388c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:18.314267 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:39:18.328635 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad174f-1f2c-4f55-8536-20be2a2388c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:39:18.330084 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad174f-1f2c-4f55-8536-20be2a2388c5 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:39:18.348501 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 661/1322] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:39:18 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:39:21.102742 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecf995-bf17-4e24-8cc5-90d092ee7eda None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:21.103329 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecf995-bf17-4e24-8cc5-90d092ee7eda None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:21.103329 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecf995-bf17-4e24-8cc5-90d092ee7eda None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:21.104707 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 662/1323] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:39:21.110013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913844a-aa20-4aed-8cce-a7d08f818225 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:21.110435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913844a-aa20-4aed-8cce-a7d08f818225 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:21.110522 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913844a-aa20-4aed-8cce-a7d08f818225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:21.177945 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-9913844a-aa20-4aed-8cce-a7d08f818225 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:39:21.179177 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-9913844a-aa20-4aed-8cce-a7d08f818225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:21.288871 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 662/1324] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:39:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:39:21.326355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ecf995-bf17-4e24-8cc5-90d092ee7eda None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:39:21.327952 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-79ecf995-bf17-4e24-8cc5-90d092ee7eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:21.348449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:39:21.441209 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ecf995-bf17-4e24-8cc5-90d092ee7eda None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:39:21.519805 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37662597-1519-4e35-894e-c3d09d720e5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:39:21.520468 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37662597-1519-4e35-894e-c3d09d720e5f service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:21.520840 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37662597-1519-4e35-894e-c3d09d720e5f service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:21.521118 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37662597-1519-4e35-894e-c3d09d720e5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:21.522757 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-37662597-1519-4e35-894e-c3d09d720e5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:21.594332 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:39:21.632860 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37662597-1519-4e35-894e-c3d09d720e5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:39:21.634293 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37662597-1519-4e35-894e-c3d09d720e5f service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:39:21.653758 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 663/1325] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:39:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:39:23.736718 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da94a87-4250-4873-8ed0-f5197b868ee0 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:23.737226 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da94a87-4250-4873-8ed0-f5197b868ee0 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:23.737226 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da94a87-4250-4873-8ed0-f5197b868ee0 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:23.738798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 663/1326] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:39:23.744194 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c030d4d-66d8-4626-8a36-4b95191877c4 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:23.744630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c030d4d-66d8-4626-8a36-4b95191877c4 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:23.744727 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c030d4d-66d8-4626-8a36-4b95191877c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:23.809777 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-7c030d4d-66d8-4626-8a36-4b95191877c4 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:39:23.810982 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-7c030d4d-66d8-4626-8a36-4b95191877c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:23.908732 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 664/1327] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:39:23.944964 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da94a87-4250-4873-8ed0-f5197b868ee0 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:39:23.946782 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-8da94a87-4250-4873-8ed0-f5197b868ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:24.008682 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:39:24.054043 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da94a87-4250-4873-8ed0-f5197b868ee0 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:39:24.134443 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3d85a-57d7-4b44-a0d8-0bc4becdbad1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:39:24.135460 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3d85a-57d7-4b44-a0d8-0bc4becdbad1 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:24.135960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3d85a-57d7-4b44-a0d8-0bc4becdbad1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:24.136232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3d85a-57d7-4b44-a0d8-0bc4becdbad1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:24.137882 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-f6f3d85a-57d7-4b44-a0d8-0bc4becdbad1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:24.252997 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:39:24.255494 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f3d85a-57d7-4b44-a0d8-0bc4becdbad1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:39:24.257711 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f3d85a-57d7-4b44-a0d8-0bc4becdbad1 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:39:24.276303 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 664/1328] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:39:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:39:26.345559 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78040be3-5a28-455b-9988-355b215c6449 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:26.346185 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78040be3-5a28-455b-9988-355b215c6449 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:26.346185 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78040be3-5a28-455b-9988-355b215c6449 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:26.347525 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 665/1329] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:39:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:39:26.353144 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e5104-ae0b-4bcd-b950-8666de3440fd None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:26.353565 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e5104-ae0b-4bcd-b950-8666de3440fd None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:26.353838 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e5104-ae0b-4bcd-b950-8666de3440fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:26.421414 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-600e5104-ae0b-4bcd-b950-8666de3440fd None None] MFA Rules not processed for user `55e7a5b2a35b46a0a3eedafbe6c53a2c`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:39:26.422715 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-600e5104-ae0b-4bcd-b950-8666de3440fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:26.527107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 665/1330] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:39:26 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:39:26.565044 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78040be3-5a28-455b-9988-355b215c6449 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:39:26.566636 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-78040be3-5a28-455b-9988-355b215c6449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:26.598854 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:39:26.677183 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78040be3-5a28-455b-9988-355b215c6449 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:39:26.756541 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1926e1a-4026-4a0f-915d-994d5471ef7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:39:26.757281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1926e1a-4026-4a0f-915d-994d5471ef7b service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:26.757641 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1926e1a-4026-4a0f-915d-994d5471ef7b service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:26.757898 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1926e1a-4026-4a0f-915d-994d5471ef7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:26.759628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-a1926e1a-4026-4a0f-915d-994d5471ef7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:26.838450 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:39:26.871826 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1926e1a-4026-4a0f-915d-994d5471ef7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:39:26.873636 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1926e1a-4026-4a0f-915d-994d5471ef7b service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:39:26.900921 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 666/1331] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:39:26 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:39:29.181440 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210865f9-56c5-405a-b6e9-f0793afb134a None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:29.182272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210865f9-56c5-405a-b6e9-f0793afb134a None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:29.182380 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210865f9-56c5-405a-b6e9-f0793afb134a None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:29.184650 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 666/1332] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:39:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:39:29.190785 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd884b14-2006-4946-b09e-fdd8456f34e6 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:29.191556 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd884b14-2006-4946-b09e-fdd8456f34e6 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:29.191756 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd884b14-2006-4946-b09e-fdd8456f34e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:29.260733 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-fd884b14-2006-4946-b09e-fdd8456f34e6 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:39:29.261847 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-fd884b14-2006-4946-b09e-fdd8456f34e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:29.368161 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 667/1333] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:39:29.410839 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210865f9-56c5-405a-b6e9-f0793afb134a None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:39:29.412544 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-210865f9-56c5-405a-b6e9-f0793afb134a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:29.484481 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:39:29.528050 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210865f9-56c5-405a-b6e9-f0793afb134a None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:39:29.612335 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6fe1b-4f6c-48bb-82b4-0c874bfb7f7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:39:29.613156 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6fe1b-4f6c-48bb-82b4-0c874bfb7f7a service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:29.613522 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6fe1b-4f6c-48bb-82b4-0c874bfb7f7a service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:29.613889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6fe1b-4f6c-48bb-82b4-0c874bfb7f7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:29.615909 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-6ab6fe1b-4f6c-48bb-82b4-0c874bfb7f7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:29.739272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab6fe1b-4f6c-48bb-82b4-0c874bfb7f7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:39:29.740742 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab6fe1b-4f6c-48bb-82b4-0c874bfb7f7a service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:39:29.750838 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:39:29.759816 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 667/1334] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:39:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:39:31.808404 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966becb-d08e-4e85-ab5c-e51702b92576 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:31.809254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966becb-d08e-4e85-ab5c-e51702b92576 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:31.809254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966becb-d08e-4e85-ab5c-e51702b92576 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:31.811601 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 668/1335] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:39:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:39:31.817765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8acb01-319e-45d0-8069-b480c406cc49 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:31.818189 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8acb01-319e-45d0-8069-b480c406cc49 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:31.818328 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8acb01-319e-45d0-8069-b480c406cc49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:31.887998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-ca8acb01-319e-45d0-8069-b480c406cc49 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:39:31.889210 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-ca8acb01-319e-45d0-8069-b480c406cc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:31.987220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 668/1336] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:39:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:39:32.030730 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966becb-d08e-4e85-ab5c-e51702b92576 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:39:32.033325 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-4966becb-d08e-4e85-ab5c-e51702b92576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:32.092901 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:39:32.168035 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966becb-d08e-4e85-ab5c-e51702b92576 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:39:32.256765 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c695d3e-aac9-4a5b-86c5-fd2be177118a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:39:32.257427 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c695d3e-aac9-4a5b-86c5-fd2be177118a service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:32.257715 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c695d3e-aac9-4a5b-86c5-fd2be177118a service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:32.257991 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c695d3e-aac9-4a5b-86c5-fd2be177118a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:32.259700 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-7c695d3e-aac9-4a5b-86c5-fd2be177118a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:32.347644 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:39:32.385880 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c695d3e-aac9-4a5b-86c5-fd2be177118a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:39:32.387375 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c695d3e-aac9-4a5b-86c5-fd2be177118a service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:39:32.405634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 669/1337] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:39:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:39:34.879337 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d419ac-4989-4df7-95ec-1ed186c94597 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:34.880225 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d419ac-4989-4df7-95ec-1ed186c94597 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:34.880225 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d419ac-4989-4df7-95ec-1ed186c94597 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:34.882363 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 669/1338] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:39:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:39:34.891008 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c8088-8c55-4115-864c-aa355b6eb0f2 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:34.891446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c8088-8c55-4115-864c-aa355b6eb0f2 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:34.891734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c8088-8c55-4115-864c-aa355b6eb0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:34.963603 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-288c8088-8c55-4115-864c-aa355b6eb0f2 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:39:34.964829 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-288c8088-8c55-4115-864c-aa355b6eb0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:35.083702 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 670/1339] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:39:34 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:39:35.133536 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d419ac-4989-4df7-95ec-1ed186c94597 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:39:35.135196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-71d419ac-4989-4df7-95ec-1ed186c94597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:35.214789 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:39:35.246426 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d419ac-4989-4df7-95ec-1ed186c94597 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:39:35.329722 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892f11b-404c-4249-9c24-b1785efa72e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:39:35.330371 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892f11b-404c-4249-9c24-b1785efa72e6 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:35.330655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892f11b-404c-4249-9c24-b1785efa72e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:35.330932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892f11b-404c-4249-9c24-b1785efa72e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:35.332624 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-6892f11b-404c-4249-9c24-b1785efa72e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:35.456578 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6892f11b-404c-4249-9c24-b1785efa72e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:39:35.458632 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6892f11b-404c-4249-9c24-b1785efa72e6 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:39:35.484789 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 670/1340] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:39:35 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:39:39.490108 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c95af5-55fe-4f1b-b816-7aee1edf3153 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:39.490566 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c95af5-55fe-4f1b-b816-7aee1edf3153 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:39.490664 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c95af5-55fe-4f1b-b816-7aee1edf3153 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:39.492115 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 671/1341] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:39:39.497474 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc2eec-55cc-455f-a7db-629cd084aa13 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:39.497917 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc2eec-55cc-455f-a7db-629cd084aa13 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:39.498016 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc2eec-55cc-455f-a7db-629cd084aa13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:39.502541 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:39:39.569283 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-29bc2eec-55cc-455f-a7db-629cd084aa13 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:39:39.570512 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-29bc2eec-55cc-455f-a7db-629cd084aa13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:39.677820 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 671/1342] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:39:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:39:39.717142 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c95af5-55fe-4f1b-b816-7aee1edf3153 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:39:39.718771 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-53c95af5-55fe-4f1b-b816-7aee1edf3153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:39.771914 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:39:39.830974 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c95af5-55fe-4f1b-b816-7aee1edf3153 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:39:39.906565 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce666c1-53ac-41ca-9d70-0ffded4bf54a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:39:39.907258 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce666c1-53ac-41ca-9d70-0ffded4bf54a service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:39.907543 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce666c1-53ac-41ca-9d70-0ffded4bf54a service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:39.907824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce666c1-53ac-41ca-9d70-0ffded4bf54a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:39.909493 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-9ce666c1-53ac-41ca-9d70-0ffded4bf54a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:40.008675 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:39:40.026376 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce666c1-53ac-41ca-9d70-0ffded4bf54a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:39:40.027797 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce666c1-53ac-41ca-9d70-0ffded4bf54a service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:39:40.046687 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 672/1343] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:39:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:39:42.807289 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a16326-e4b7-42c9-b4cf-9c572b7850b1 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:42.807842 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a16326-e4b7-42c9-b4cf-9c572b7850b1 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:42.807842 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a16326-e4b7-42c9-b4cf-9c572b7850b1 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:42.809316 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 672/1344] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:39:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:39:42.814933 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9e6ea-67bb-4f99-8798-cda32468191f None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:42.815369 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9e6ea-67bb-4f99-8798-cda32468191f None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:42.815558 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9e6ea-67bb-4f99-8798-cda32468191f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:42.882946 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-b2d9e6ea-67bb-4f99-8798-cda32468191f None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:39:42.884242 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-b2d9e6ea-67bb-4f99-8798-cda32468191f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:42.983171 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 673/1345] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:39:42 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:39:43.021318 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a16326-e4b7-42c9-b4cf-9c572b7850b1 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:39:43.022929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-74a16326-e4b7-42c9-b4cf-9c572b7850b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:43.113259 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:39:43.130637 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a16326-e4b7-42c9-b4cf-9c572b7850b1 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:39:43.209380 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f93cd02-e3e5-40d6-8c92-9f7500b60268 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:39:43.210047 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93cd02-e3e5-40d6-8c92-9f7500b60268 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:43.210360 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93cd02-e3e5-40d6-8c92-9f7500b60268 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:43.210599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93cd02-e3e5-40d6-8c92-9f7500b60268 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:43.212313 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-9f93cd02-e3e5-40d6-8c92-9f7500b60268 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:43.336202 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f93cd02-e3e5-40d6-8c92-9f7500b60268 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:39:43.337661 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f93cd02-e3e5-40d6-8c92-9f7500b60268 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:39:43.356707 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 673/1346] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:39:43 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:39:47.553734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add39e9d-68f7-4f49-8b16-4278791f7f99 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:47.554294 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add39e9d-68f7-4f49-8b16-4278791f7f99 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:47.554294 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add39e9d-68f7-4f49-8b16-4278791f7f99 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:47.555727 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 674/1347] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:39:47.561254 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dabeb7-7677-4f26-a9ac-f6f9a2c99eba None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:47.561829 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dabeb7-7677-4f26-a9ac-f6f9a2c99eba None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:47.561940 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dabeb7-7677-4f26-a9ac-f6f9a2c99eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:47.577542 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:39:47.633044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-70dabeb7-7677-4f26-a9ac-f6f9a2c99eba None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:39:47.634198 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-70dabeb7-7677-4f26-a9ac-f6f9a2c99eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:47.751100 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 674/1348] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:39:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:39:47.778487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add39e9d-68f7-4f49-8b16-4278791f7f99 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:39:47.780296 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-add39e9d-68f7-4f49-8b16-4278791f7f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:47.849377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:39:47.903419 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add39e9d-68f7-4f49-8b16-4278791f7f99 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:39:47.986574 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5b963-4cf3-4ec4-b0ff-57a0c5f61f5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:39:47.987276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5b963-4cf3-4ec4-b0ff-57a0c5f61f5b service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:47.987560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5b963-4cf3-4ec4-b0ff-57a0c5f61f5b service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:47.987838 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5b963-4cf3-4ec4-b0ff-57a0c5f61f5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:47.989917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-15f5b963-4cf3-4ec4-b0ff-57a0c5f61f5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:48.101903 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:39:48.106259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f5b963-4cf3-4ec4-b0ff-57a0c5f61f5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:39:48.107843 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f5b963-4cf3-4ec4-b0ff-57a0c5f61f5b service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:39:48.126596 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 675/1349] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:39:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:39:52.985411 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84539b2d-aaee-4107-b7e8-41308e763166 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:52.985899 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84539b2d-aaee-4107-b7e8-41308e763166 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:52.985899 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84539b2d-aaee-4107-b7e8-41308e763166 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:52.987395 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 675/1350] 10.222.0.15 () {54 vars in 921 bytes} [Wed Jul 20 14:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:39:52.992597 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adb418-2f1e-4a5a-b279-d2aa45da5b34 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:52.993007 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adb418-2f1e-4a5a-b279-d2aa45da5b34 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:53.004755 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adb418-2f1e-4a5a-b279-d2aa45da5b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:53.114619 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-86adb418-2f1e-4a5a-b279-d2aa45da5b34 None None] MFA Rules not processed for user `cb857f32bbc04d82bfa681bd5659765f`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:39:53.115836 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-86adb418-2f1e-4a5a-b279-d2aa45da5b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:53.239717 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 676/1351] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Jul 20 14:39:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:39:57.295762 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34692c7e-73ed-4fb3-8671-469660c309e5 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:57.296375 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34692c7e-73ed-4fb3-8671-469660c309e5 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:57.296375 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34692c7e-73ed-4fb3-8671-469660c309e5 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:57.297955 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 676/1352] 10.222.0.15 () {54 vars in 921 bytes} [Wed Jul 20 14:39:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:39:57.303323 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85b05f-6fcd-481b-852e-e37c1d57d8ae None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:57.303772 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85b05f-6fcd-481b-852e-e37c1d57d8ae None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:57.303878 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85b05f-6fcd-481b-852e-e37c1d57d8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:57.357997 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:39:57.384327 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-9e85b05f-6fcd-481b-852e-e37c1d57d8ae None None] MFA Rules not processed for user `cb857f32bbc04d82bfa681bd5659765f`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:39:57.385550 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-9e85b05f-6fcd-481b-852e-e37c1d57d8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:57.498350 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 677/1353] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Jul 20 14:39:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:39:57.738386 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=59 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:39:58.945897 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684651e-3ca0-402b-b8ec-630c5b6b5d12 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:58.946573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684651e-3ca0-402b-b8ec-630c5b6b5d12 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:58.946573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684651e-3ca0-402b-b8ec-630c5b6b5d12 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:58.948061 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 677/1354] 10.222.0.15 () {54 vars in 921 bytes} [Wed Jul 20 14:39:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:39:58.953365 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437dc29a-ad73-4aa4-a88d-c95812b5783a None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:39:58.953811 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437dc29a-ad73-4aa4-a88d-c95812b5783a None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:39:58.953935 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437dc29a-ad73-4aa4-a88d-c95812b5783a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:39:59.031870 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-437dc29a-ad73-4aa4-a88d-c95812b5783a None None] MFA Rules not processed for user `cb857f32bbc04d82bfa681bd5659765f`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:39:59.033096 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-437dc29a-ad73-4aa4-a88d-c95812b5783a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:39:59.078213 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:39:59.149622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 678/1355] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Jul 20 14:39:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:40:02.597409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa21693-0fd4-48c3-a073-260b80facc5d None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:02.598799 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa21693-0fd4-48c3-a073-260b80facc5d None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:02.599274 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa21693-0fd4-48c3-a073-260b80facc5d None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:02.600862 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 678/1356] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:40:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:40:02.606508 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa757c4-245d-4321-9f92-bf4422167dfe None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:02.606941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa757c4-245d-4321-9f92-bf4422167dfe None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:02.607084 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa757c4-245d-4321-9f92-bf4422167dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:02.679831 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-4fa757c4-245d-4321-9f92-bf4422167dfe None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:40:02.681108 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-4fa757c4-245d-4321-9f92-bf4422167dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:02.788395 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 679/1357] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:40:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:40:02.844587 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b79fe7-a90f-4b93-8212-2f38cc71a106 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:02.844881 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b79fe7-a90f-4b93-8212-2f38cc71a106 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:02.845191 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b79fe7-a90f-4b93-8212-2f38cc71a106 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:02.846584 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 679/1358] 10.222.0.15 () {54 vars in 962 bytes} [Wed Jul 20 14:40:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:40:02.857666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60e82d-9590-4e5a-a8a5-ed164154c703 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:02.858059 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60e82d-9590-4e5a-a8a5-ed164154c703 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:02.858419 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60e82d-9590-4e5a-a8a5-ed164154c703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:02.878900 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:02.945317 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-5f60e82d-9590-4e5a-a8a5-ed164154c703 None None] MFA Rules not processed for user `5d4a107ed04348b59d20d1703dbe04eb`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:40:02.946625 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-5f60e82d-9590-4e5a-a8a5-ed164154c703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:03.070582 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 680/1359] 10.222.0.15 () {58 vars in 1059 bytes} [Wed Jul 20 14:40:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:40:03.078414 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b79fe7-a90f-4b93-8212-2f38cc71a106 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:40:03.080324 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-d6b79fe7-a90f-4b93-8212-2f38cc71a106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:03.237849 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b79fe7-a90f-4b93-8212-2f38cc71a106 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:40:03.316664 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:03.358371 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a780e0b-7c5c-4d70-b584-509c2140919e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:40:03.359135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a780e0b-7c5c-4d70-b584-509c2140919e service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:03.359528 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a780e0b-7c5c-4d70-b584-509c2140919e service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:03.359916 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a780e0b-7c5c-4d70-b584-509c2140919e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:03.361653 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-2a780e0b-7c5c-4d70-b584-509c2140919e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:03.457483 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a780e0b-7c5c-4d70-b584-509c2140919e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:40:03.459718 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a780e0b-7c5c-4d70-b584-509c2140919e service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:40:03.486217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 680/1360] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:40:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 14:40:05.723379 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ae2f0-4720-4a6b-849d-a6ec4281d745 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:05.723986 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ae2f0-4720-4a6b-849d-a6ec4281d745 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:05.723986 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ae2f0-4720-4a6b-849d-a6ec4281d745 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:05.725519 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 681/1361] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:40:05.730865 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f677c44-82e8-4c74-915b-723182be035a None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:05.731351 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f677c44-82e8-4c74-915b-723182be035a None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:05.731502 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f677c44-82e8-4c74-915b-723182be035a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:05.801432 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-2f677c44-82e8-4c74-915b-723182be035a None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:40:05.802798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-2f677c44-82e8-4c74-915b-723182be035a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:05.843522 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:05.915090 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 681/1362] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:40:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:40:05.956669 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ae2f0-4720-4a6b-849d-a6ec4281d745 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:40:05.959356 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-be0ae2f0-4720-4a6b-849d-a6ec4281d745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:06.046062 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:06.144468 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ae2f0-4720-4a6b-849d-a6ec4281d745 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:40:06.290997 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a9b909-5a58-41c9-9986-8ad0705cd33e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:40:06.291689 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9b909-5a58-41c9-9986-8ad0705cd33e service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:06.292063 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9b909-5a58-41c9-9986-8ad0705cd33e service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:06.292341 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9b909-5a58-41c9-9986-8ad0705cd33e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:06.294133 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-61a9b909-5a58-41c9-9986-8ad0705cd33e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:06.316751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:06.386201 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a9b909-5a58-41c9-9986-8ad0705cd33e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:40:06.387835 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a9b909-5a58-41c9-9986-8ad0705cd33e service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:40:06.406513 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 682/1363] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:40:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:40:08.742424 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351dd822-c052-4bdd-ac7c-5b775755f8c4 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:08.742937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351dd822-c052-4bdd-ac7c-5b775755f8c4 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:08.742937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351dd822-c052-4bdd-ac7c-5b775755f8c4 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:08.744310 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 682/1364] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:40:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:40:08.749787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f085e6-a65f-44d1-bc6c-889910b4dfc9 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:08.750228 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f085e6-a65f-44d1-bc6c-889910b4dfc9 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:08.750353 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f085e6-a65f-44d1-bc6c-889910b4dfc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:08.834274 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-b7f085e6-a65f-44d1-bc6c-889910b4dfc9 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:40:08.835512 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-b7f085e6-a65f-44d1-bc6c-889910b4dfc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:08.938571 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:08.943096 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 683/1365] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:40:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:40:08.988101 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195df77-12f0-49ea-898b-11b03be8515c None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:08.988400 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195df77-12f0-49ea-898b-11b03be8515c None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:08.988609 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195df77-12f0-49ea-898b-11b03be8515c None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:08.990280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 683/1366] 10.222.0.15 () {54 vars in 962 bytes} [Wed Jul 20 14:40:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:40:08.996435 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc27182-bc9b-47eb-a44d-343c30325923 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:08.996845 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc27182-bc9b-47eb-a44d-343c30325923 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:08.997193 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc27182-bc9b-47eb-a44d-343c30325923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:09.074967 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-bfc27182-bc9b-47eb-a44d-343c30325923 None None] MFA Rules not processed for user `5d4a107ed04348b59d20d1703dbe04eb`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:40:09.076208 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-bfc27182-bc9b-47eb-a44d-343c30325923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:09.198890 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 684/1367] 10.222.0.15 () {58 vars in 1059 bytes} [Wed Jul 20 14:40:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:40:09.207276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0195df77-12f0-49ea-898b-11b03be8515c None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:40:09.208920 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-0195df77-12f0-49ea-898b-11b03be8515c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:09.368779 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0195df77-12f0-49ea-898b-11b03be8515c None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:40:09.391166 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:09.496965 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14407588-139e-4afa-9a36-9f91062bf876 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:40:09.497754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14407588-139e-4afa-9a36-9f91062bf876 service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:09.498263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14407588-139e-4afa-9a36-9f91062bf876 service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:09.498635 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14407588-139e-4afa-9a36-9f91062bf876 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:09.500433 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-14407588-139e-4afa-9a36-9f91062bf876 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:09.600165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14407588-139e-4afa-9a36-9f91062bf876 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:40:09.602130 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14407588-139e-4afa-9a36-9f91062bf876 service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:40:09.623829 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 684/1368] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:40:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:40:10.656165 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a375ba-8e15-4eb0-80c4-2c13d0d7ebe3 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:10.656705 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a375ba-8e15-4eb0-80c4-2c13d0d7ebe3 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:10.657139 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a375ba-8e15-4eb0-80c4-2c13d0d7ebe3 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:10.658828 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 685/1369] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:40:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:40:10.664622 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f342c-6a55-4511-8f07-dc32dd16a56b None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:10.665058 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f342c-6a55-4511-8f07-dc32dd16a56b None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:10.665138 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f342c-6a55-4511-8f07-dc32dd16a56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:10.697525 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:10.735129 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-d92f342c-6a55-4511-8f07-dc32dd16a56b None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:40:10.736342 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-d92f342c-6a55-4511-8f07-dc32dd16a56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:10.840541 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 685/1370] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:40:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:40:11.179753 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b89d94-d254-4e8f-acc5-e002dc3c0288 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:11.179753 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b89d94-d254-4e8f-acc5-e002dc3c0288 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:11.179753 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b89d94-d254-4e8f-acc5-e002dc3c0288 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:11.182709 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 686/1371] 10.222.0.15 () {54 vars in 964 bytes} [Wed Jul 20 14:40:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:40:11.188048 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a01f79-c030-4cfd-9e7c-08426291965e None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:11.188450 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a01f79-c030-4cfd-9e7c-08426291965e None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:11.188828 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a01f79-c030-4cfd-9e7c-08426291965e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:11.275078 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-02a01f79-c030-4cfd-9e7c-08426291965e None None] MFA Rules not processed for user `9618ec9bbe76431e84640b0e5ca90eb1`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:40:11.276467 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-02a01f79-c030-4cfd-9e7c-08426291965e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:11.327994 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:11.361680 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 686/1372] 10.222.0.15 () {58 vars in 1061 bytes} [Wed Jul 20 14:40:11 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:40:11.369911 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b89d94-d254-4e8f-acc5-e002dc3c0288 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:40:11.371572 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-84b89d94-d254-4e8f-acc5-e002dc3c0288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:11.430209 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:11.494606 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b89d94-d254-4e8f-acc5-e002dc3c0288 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:40:11.573789 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb2bd59-f4ad-4d58-9b28-e6563bdf310d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:40:11.574641 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2bd59-f4ad-4d58-9b28-e6563bdf310d service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:11.575021 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2bd59-f4ad-4d58-9b28-e6563bdf310d service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:11.575434 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2bd59-f4ad-4d58-9b28-e6563bdf310d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:11.577190 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-beb2bd59-f4ad-4d58-9b28-e6563bdf310d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:11.683481 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:11.700866 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb2bd59-f4ad-4d58-9b28-e6563bdf310d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:40:11.702657 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb2bd59-f4ad-4d58-9b28-e6563bdf310d service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:40:11.706838 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2bc4a-9b2e-46ec-a558-11c08806e3ae None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:11.707364 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2bc4a-9b2e-46ec-a558-11c08806e3ae None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:11.707667 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2bc4a-9b2e-46ec-a558-11c08806e3ae None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:11.712898 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 687/1373] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:40:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 14:40:11.713984 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972f965-c3d7-494f-a5a7-a55ce5226ed9 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:11.714275 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972f965-c3d7-494f-a5a7-a55ce5226ed9 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:11.714528 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972f965-c3d7-494f-a5a7-a55ce5226ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:11.727442 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 687/1374] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:40:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:40:11.788889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-4972f965-c3d7-494f-a5a7-a55ce5226ed9 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:40:11.790106 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-4972f965-c3d7-494f-a5a7-a55ce5226ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:11.885161 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 688/1375] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:40:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:40:11.931676 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb2bd59-f4ad-4d58-9b28-e6563bdf310d service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:40:11.931676 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-beb2bd59-f4ad-4d58-9b28-e6563bdf310d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:12.041052 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb2bd59-f4ad-4d58-9b28-e6563bdf310d service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:40:12.160019 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:12.172100 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6aa81-2a9a-48ad-81c8-32caad951625 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:40:12.172755 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6aa81-2a9a-48ad-81c8-32caad951625 service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:12.173057 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6aa81-2a9a-48ad-81c8-32caad951625 service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:12.173332 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6aa81-2a9a-48ad-81c8-32caad951625 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:12.175005 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e6b6aa81-2a9a-48ad-81c8-32caad951625 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:12.264631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b6aa81-2a9a-48ad-81c8-32caad951625 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:40:12.266189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b6aa81-2a9a-48ad-81c8-32caad951625 service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:40:12.284094 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 688/1376] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:40:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:40:14.341293 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306829e-2c2b-4077-b4e8-d4a010b13636 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:14.341806 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306829e-2c2b-4077-b4e8-d4a010b13636 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:14.341806 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306829e-2c2b-4077-b4e8-d4a010b13636 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:14.343133 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 689/1377] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:40:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:40:14.348541 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b6a3db-c5d3-4679-a77d-cfe786f1a65f None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:14.348980 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b6a3db-c5d3-4679-a77d-cfe786f1a65f None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:14.349178 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b6a3db-c5d3-4679-a77d-cfe786f1a65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:14.416998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-b3b6a3db-c5d3-4679-a77d-cfe786f1a65f None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:40:14.418241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-b3b6a3db-c5d3-4679-a77d-cfe786f1a65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:14.486399 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:14.528672 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 689/1378] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:40:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:40:14.573167 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0306829e-2c2b-4077-b4e8-d4a010b13636 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:40:14.574695 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-0306829e-2c2b-4077-b4e8-d4a010b13636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:14.634398 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:14.745232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0306829e-2c2b-4077-b4e8-d4a010b13636 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:40:14.863334 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9348c244-cff2-4683-b14a-d8633b6de5d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:40:14.863998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348c244-cff2-4683-b14a-d8633b6de5d2 service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:14.864273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348c244-cff2-4683-b14a-d8633b6de5d2 service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:14.864552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348c244-cff2-4683-b14a-d8633b6de5d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:14.866193 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-9348c244-cff2-4683-b14a-d8633b6de5d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:14.878218 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:14.961752 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9348c244-cff2-4683-b14a-d8633b6de5d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:40:14.963403 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9348c244-cff2-4683-b14a-d8633b6de5d2 service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:40:14.982124 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 690/1379] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:40:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:40:17.075260 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24718319-082d-4cc5-9891-bc42971125eb None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:17.075775 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24718319-082d-4cc5-9891-bc42971125eb None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:17.075775 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24718319-082d-4cc5-9891-bc42971125eb None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:17.077297 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 690/1380] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:40:17.082724 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca8dce-a20e-423a-befd-5e6520596af8 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:17.083244 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca8dce-a20e-423a-befd-5e6520596af8 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:17.083344 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca8dce-a20e-423a-befd-5e6520596af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:17.158971 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-dbca8dce-a20e-423a-befd-5e6520596af8 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:40:17.160794 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-dbca8dce-a20e-423a-befd-5e6520596af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:17.261625 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:17.275481 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 691/1381] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:40:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:40:17.319996 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24718319-082d-4cc5-9891-bc42971125eb None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:40:17.321716 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-24718319-082d-4cc5-9891-bc42971125eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:17.469588 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24718319-082d-4cc5-9891-bc42971125eb None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:40:17.519147 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:17.590134 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86de902-9b54-4bec-8591-7b2aa0c2fe55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:40:17.590927 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86de902-9b54-4bec-8591-7b2aa0c2fe55 service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:17.591342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86de902-9b54-4bec-8591-7b2aa0c2fe55 service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:17.591693 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86de902-9b54-4bec-8591-7b2aa0c2fe55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:17.593436 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-a86de902-9b54-4bec-8591-7b2aa0c2fe55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:17.690615 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86de902-9b54-4bec-8591-7b2aa0c2fe55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:40:17.692156 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86de902-9b54-4bec-8591-7b2aa0c2fe55 service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:40:17.711677 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 691/1382] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:40:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:40:19.886634 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff1348-1f32-48de-8351-14f58d48557d None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:19.887368 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff1348-1f32-48de-8351-14f58d48557d None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:19.887368 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff1348-1f32-48de-8351-14f58d48557d None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:19.888982 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 692/1383] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:40:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:40:19.895929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd1c6d-4657-435b-95c5-6d7b14993dff None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:19.896502 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd1c6d-4657-435b-95c5-6d7b14993dff None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:19.896502 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd1c6d-4657-435b-95c5-6d7b14993dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:19.970182 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-e4cd1c6d-4657-435b-95c5-6d7b14993dff None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:40:19.971451 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e4cd1c6d-4657-435b-95c5-6d7b14993dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:19.973555 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:20.084447 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 692/1384] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:40:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:40:20.129186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff1348-1f32-48de-8351-14f58d48557d None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:40:20.130791 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-62ff1348-1f32-48de-8351-14f58d48557d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:20.300248 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff1348-1f32-48de-8351-14f58d48557d None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:40:20.368901 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:20.423913 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fdc6b2-8252-4f47-be4a-5edd41c25d3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:40:20.424669 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fdc6b2-8252-4f47-be4a-5edd41c25d3d service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:20.425021 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fdc6b2-8252-4f47-be4a-5edd41c25d3d service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:20.425327 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fdc6b2-8252-4f47-be4a-5edd41c25d3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:20.427003 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-71fdc6b2-8252-4f47-be4a-5edd41c25d3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:20.526799 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fdc6b2-8252-4f47-be4a-5edd41c25d3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:40:20.528454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fdc6b2-8252-4f47-be4a-5edd41c25d3d service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:40:20.546653 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 693/1385] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:40:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:40:22.642733 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892da842-eeb4-422a-89f8-f27ddd682ddf None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:22.643485 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892da842-eeb4-422a-89f8-f27ddd682ddf None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:22.643485 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892da842-eeb4-422a-89f8-f27ddd682ddf None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:22.644748 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 693/1386] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:40:22.650016 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea951e83-ff4b-4f80-a642-319e1aa598e7 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:22.650532 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea951e83-ff4b-4f80-a642-319e1aa598e7 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:22.650708 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea951e83-ff4b-4f80-a642-319e1aa598e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:22.720830 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-ea951e83-ff4b-4f80-a642-319e1aa598e7 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:40:22.721986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-ea951e83-ff4b-4f80-a642-319e1aa598e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:22.750230 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:22.833275 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 694/1387] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:40:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:40:22.879430 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892da842-eeb4-422a-89f8-f27ddd682ddf None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:40:22.881671 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-892da842-eeb4-422a-89f8-f27ddd682ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:23.029964 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:23.052290 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892da842-eeb4-422a-89f8-f27ddd682ddf None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:40:23.179596 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3fd29a-6221-4f42-a409-c5985d33ab7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:40:23.180214 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fd29a-6221-4f42-a409-c5985d33ab7c service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:23.180491 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fd29a-6221-4f42-a409-c5985d33ab7c service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:23.180770 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fd29a-6221-4f42-a409-c5985d33ab7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:23.182443 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-6f3fd29a-6221-4f42-a409-c5985d33ab7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:23.287611 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3fd29a-6221-4f42-a409-c5985d33ab7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:40:23.289734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3fd29a-6221-4f42-a409-c5985d33ab7c service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:40:23.296901 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:23.321142 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 694/1388] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:40:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 14:40:25.499825 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3237d0-d2b7-4204-b074-bdf224242611 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:25.500345 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3237d0-d2b7-4204-b074-bdf224242611 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:25.500345 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3237d0-d2b7-4204-b074-bdf224242611 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:25.502214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 695/1389] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:40:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:40:25.508079 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86698b97-90b0-4215-9103-3efbdb21e379 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:25.508545 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86698b97-90b0-4215-9103-3efbdb21e379 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:25.508774 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86698b97-90b0-4215-9103-3efbdb21e379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:25.575846 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-86698b97-90b0-4215-9103-3efbdb21e379 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:40:25.577221 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-86698b97-90b0-4215-9103-3efbdb21e379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:25.679408 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 695/1390] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:40:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:40:25.723696 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3237d0-d2b7-4204-b074-bdf224242611 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:40:25.725435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-6c3237d0-d2b7-4204-b074-bdf224242611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:25.894995 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3237d0-d2b7-4204-b074-bdf224242611 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:40:25.905035 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:26.015247 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11f2af4-2e70-4308-a3d7-20b1654a34e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:40:26.015901 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f2af4-2e70-4308-a3d7-20b1654a34e1 service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:26.016206 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f2af4-2e70-4308-a3d7-20b1654a34e1 service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:26.016435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f2af4-2e70-4308-a3d7-20b1654a34e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:26.018105 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-b11f2af4-2e70-4308-a3d7-20b1654a34e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:26.111276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11f2af4-2e70-4308-a3d7-20b1654a34e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:40:26.112951 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11f2af4-2e70-4308-a3d7-20b1654a34e1 service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:40:26.134843 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 696/1391] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:40:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:40:27.262651 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918769fa-fbed-4927-a77e-609c795e072f None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:27.263332 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918769fa-fbed-4927-a77e-609c795e072f None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:27.263438 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918769fa-fbed-4927-a77e-609c795e072f None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:27.265606 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 696/1392] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:40:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:40:27.271801 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f7692-eb2b-4eaf-ae03-e28e7f29dafb None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:27.272409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f7692-eb2b-4eaf-ae03-e28e7f29dafb None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:27.272409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f7692-eb2b-4eaf-ae03-e28e7f29dafb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:27.288408 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:27.342202 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-ba1f7692-eb2b-4eaf-ae03-e28e7f29dafb None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:40:27.343442 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-ba1f7692-eb2b-4eaf-ae03-e28e7f29dafb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:27.446614 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 697/1393] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:40:27.503575 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a0626-24f7-4f08-b0a3-c63172737561 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:27.503870 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a0626-24f7-4f08-b0a3-c63172737561 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:27.504156 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a0626-24f7-4f08-b0a3-c63172737561 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:27.505538 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 697/1394] 10.222.0.15 () {54 vars in 966 bytes} [Wed Jul 20 14:40:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:40:27.510748 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebea620-d3f5-42e0-8b37-7e3460475264 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:27.511038 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebea620-d3f5-42e0-8b37-7e3460475264 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:27.511466 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebea620-d3f5-42e0-8b37-7e3460475264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:27.597878 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-4ebea620-d3f5-42e0-8b37-7e3460475264 None None] MFA Rules not processed for user `15d7d59080cd4832a45dbb6c0349cf4c`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:40:27.599112 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-4ebea620-d3f5-42e0-8b37-7e3460475264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:27.637525 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:27.682884 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 698/1395] 10.222.0.15 () {58 vars in 1063 bytes} [Wed Jul 20 14:40:27 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:40:27.691176 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a0626-24f7-4f08-b0a3-c63172737561 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:40:27.692730 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-eb1a0626-24f7-4f08-b0a3-c63172737561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:27.770032 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:27.801840 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a0626-24f7-4f08-b0a3-c63172737561 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:40:27.874727 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4ac41-90f1-4f51-9e9c-a86af86b0f3b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d7d59080cd4832a45dbb6c0349cf4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:40:27.875318 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4ac41-90f1-4f51-9e9c-a86af86b0f3b service glance] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:27.875599 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4ac41-90f1-4f51-9e9c-a86af86b0f3b service glance] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:27.875870 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4ac41-90f1-4f51-9e9c-a86af86b0f3b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:27.877507 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-b8f4ac41-90f1-4f51-9e9c-a86af86b0f3b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:27.989150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4ac41-90f1-4f51-9e9c-a86af86b0f3b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:40:27.990605 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4ac41-90f1-4f51-9e9c-a86af86b0f3b service glance] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:40:28.006582 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:28.009986 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 698/1396] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:40:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:40:28.174518 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c76228-cfeb-4a4a-923f-5fe6b35113e0 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:28.174847 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c76228-cfeb-4a4a-923f-5fe6b35113e0 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:28.175349 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c76228-cfeb-4a4a-923f-5fe6b35113e0 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:28.176851 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 699/1397] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:40:28.182211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071102dc-fab1-413b-b612-c7b25352fccd None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:28.182504 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071102dc-fab1-413b-b612-c7b25352fccd None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:28.182806 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071102dc-fab1-413b-b612-c7b25352fccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:28.260855 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-071102dc-fab1-413b-b612-c7b25352fccd None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:40:28.262091 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-071102dc-fab1-413b-b612-c7b25352fccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:28.371165 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 699/1398] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:40:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:40:28.419891 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c76228-cfeb-4a4a-923f-5fe6b35113e0 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:40:28.421497 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-37c76228-cfeb-4a4a-923f-5fe6b35113e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:28.582592 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c76228-cfeb-4a4a-923f-5fe6b35113e0 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:40:28.629928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:28.705162 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92bc656-53a0-4197-9c4e-6bb7386657a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:40:28.705757 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92bc656-53a0-4197-9c4e-6bb7386657a5 service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:28.706062 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92bc656-53a0-4197-9c4e-6bb7386657a5 service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:28.706337 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92bc656-53a0-4197-9c4e-6bb7386657a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:28.708035 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-a92bc656-53a0-4197-9c4e-6bb7386657a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:28.726331 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e29b79-41a8-4eed-bf1f-b6d0ec911b91 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:28.726657 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e29b79-41a8-4eed-bf1f-b6d0ec911b91 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:28.726940 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e29b79-41a8-4eed-bf1f-b6d0ec911b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:28.798414 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-05e29b79-41a8-4eed-bf1f-b6d0ec911b91 None None] MFA Rules not processed for user `2f8cf6d908314281a3ddc9d9513d39c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:40:28.799714 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-05e29b79-41a8-4eed-bf1f-b6d0ec911b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:28.809442 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92bc656-53a0-4197-9c4e-6bb7386657a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:40:28.811104 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92bc656-53a0-4197-9c4e-6bb7386657a5 service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:40:28.829292 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:28.829905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 700/1399] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:40:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:40:28.890258 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 700/1400] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Jul 20 14:40:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:40:28.939326 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a82a77-efe9-4de4-8f04-5eaf99a1afd0 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:28.939714 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a82a77-efe9-4de4-8f04-5eaf99a1afd0 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:28.940001 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a82a77-efe9-4de4-8f04-5eaf99a1afd0 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:28.941962 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 701/1401] 10.222.0.15 () {54 vars in 943 bytes} [Wed Jul 20 14:40:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:40:28.949339 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671fed3-889e-4878-a775-5d0843b989ae None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:28.949713 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671fed3-889e-4878-a775-5d0843b989ae None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:28.950082 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671fed3-889e-4878-a775-5d0843b989ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:29.047143 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-e671fed3-889e-4878-a775-5d0843b989ae None None] MFA Rules not processed for user `37f4844988474d7a91133d77906611a5`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:40:29.048366 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e671fed3-889e-4878-a775-5d0843b989ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:29.157275 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 701/1402] 10.222.0.15 () {58 vars in 1040 bytes} [Wed Jul 20 14:40:28 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:40:29.167033 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a82a77-efe9-4de4-8f04-5eaf99a1afd0 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:40:29.168628 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-12a82a77-efe9-4de4-8f04-5eaf99a1afd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:29.217404 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:29.316122 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a82a77-efe9-4de4-8f04-5eaf99a1afd0 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:40:29.443379 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b979d-bdac-4f2b-a32b-567685901def service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f4844988474d7a91133d77906611a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:40:29.444065 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b979d-bdac-4f2b-a32b-567685901def service swift] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:29.444299 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b979d-bdac-4f2b-a32b-567685901def service swift] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:29.444592 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b979d-bdac-4f2b-a32b-567685901def service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:29.446285 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-ef3b979d-bdac-4f2b-a32b-567685901def service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:29.485212 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:29.532845 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3b979d-bdac-4f2b-a32b-567685901def service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:40:29.534583 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3b979d-bdac-4f2b-a32b-567685901def service swift] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:40:29.543271 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 702/1403] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jul 20 14:40:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 14:40:30.891785 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17ed60-0a0b-422a-8e63-c60b21e020b6 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:30.892420 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17ed60-0a0b-422a-8e63-c60b21e020b6 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:30.892420 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17ed60-0a0b-422a-8e63-c60b21e020b6 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:30.897427 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 702/1404] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:40:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:40:30.899156 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d87feb-cb17-4d69-97e2-77ff0ae5fa58 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:30.899632 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d87feb-cb17-4d69-97e2-77ff0ae5fa58 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:30.899729 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d87feb-cb17-4d69-97e2-77ff0ae5fa58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:30.975859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-29d87feb-cb17-4d69-97e2-77ff0ae5fa58 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:40:30.977071 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-29d87feb-cb17-4d69-97e2-77ff0ae5fa58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:31.085814 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 703/1405] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:40:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:40:31.129877 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17ed60-0a0b-422a-8e63-c60b21e020b6 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:40:31.131491 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-4c17ed60-0a0b-422a-8e63-c60b21e020b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:31.134826 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:31.297539 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17ed60-0a0b-422a-8e63-c60b21e020b6 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:40:31.383417 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:31.431162 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b6eb0-d63d-4876-bcfc-74e4f94dc3e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:40:31.431828 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b6eb0-d63d-4876-bcfc-74e4f94dc3e3 service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:31.432120 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b6eb0-d63d-4876-bcfc-74e4f94dc3e3 service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:31.432393 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b6eb0-d63d-4876-bcfc-74e4f94dc3e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:31.434065 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-ab9b6eb0-d63d-4876-bcfc-74e4f94dc3e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:31.541257 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9b6eb0-d63d-4876-bcfc-74e4f94dc3e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:40:31.542988 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9b6eb0-d63d-4876-bcfc-74e4f94dc3e3 service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:40:31.562960 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 703/1406] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:40:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:40:33.624443 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40dd43-dabb-44a8-a350-65547997ef75 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:33.624990 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40dd43-dabb-44a8-a350-65547997ef75 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:33.624990 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40dd43-dabb-44a8-a350-65547997ef75 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:33.626940 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 704/1407] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:40:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:40:33.632433 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb057b-608c-454c-aae4-299c9c589248 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:33.632923 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb057b-608c-454c-aae4-299c9c589248 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:33.633086 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb057b-608c-454c-aae4-299c9c589248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:33.706606 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-19fb057b-608c-454c-aae4-299c9c589248 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:40:33.708118 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-19fb057b-608c-454c-aae4-299c9c589248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:33.757598 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:33.822174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 704/1408] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:40:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:40:33.865879 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40dd43-dabb-44a8-a350-65547997ef75 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:40:33.867483 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-ca40dd43-dabb-44a8-a350-65547997ef75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:33.896404 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:34.038113 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40dd43-dabb-44a8-a350-65547997ef75 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:40:34.146243 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:34.170930 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5f7d9-9367-45b1-8310-3515d0a1a810 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:40:34.171642 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5f7d9-9367-45b1-8310-3515d0a1a810 service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:34.171929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5f7d9-9367-45b1-8310-3515d0a1a810 service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:34.172208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5f7d9-9367-45b1-8310-3515d0a1a810 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:34.173941 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-fee5f7d9-9367-45b1-8310-3515d0a1a810 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:34.276993 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee5f7d9-9367-45b1-8310-3515d0a1a810 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:40:34.278874 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee5f7d9-9367-45b1-8310-3515d0a1a810 service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:40:34.301053 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 705/1409] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:40:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:40:43.820653 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=26 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:44.298004 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=52 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:51.446134 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08fa9e-fc94-4159-9364-8ba112b8ac51 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:51.446732 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08fa9e-fc94-4159-9364-8ba112b8ac51 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:51.446732 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08fa9e-fc94-4159-9364-8ba112b8ac51 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:51.448142 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 705/1410] 10.222.0.15 () {54 vars in 918 bytes} [Wed Jul 20 14:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:40:51.453764 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69521480-7508-4559-8a83-3c844321a66e None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:51.454224 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69521480-7508-4559-8a83-3c844321a66e None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:51.454331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69521480-7508-4559-8a83-3c844321a66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:51.535306 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-69521480-7508-4559-8a83-3c844321a66e None None] MFA Rules not processed for user `cb857f32bbc04d82bfa681bd5659765f`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:40:51.536540 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-69521480-7508-4559-8a83-3c844321a66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:51.667225 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 706/1411] 10.222.0.15 () {58 vars in 1015 bytes} [Wed Jul 20 14:40:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:40:51.687486 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0354ce-d65a-4797-8ee3-584eae499f68 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:51.687772 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0354ce-d65a-4797-8ee3-584eae499f68 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:51.688059 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0354ce-d65a-4797-8ee3-584eae499f68 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:51.689474 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 706/1412] 10.222.0.15 () {54 vars in 918 bytes} [Wed Jul 20 14:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:40:51.694727 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38a434-8af8-4055-a1e0-6177ae3f15c4 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:51.695028 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38a434-8af8-4055-a1e0-6177ae3f15c4 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:51.695341 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38a434-8af8-4055-a1e0-6177ae3f15c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:51.770568 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:51.775093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-de38a434-8af8-4055-a1e0-6177ae3f15c4 None None] MFA Rules not processed for user `5d4a107ed04348b59d20d1703dbe04eb`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:40:51.778037 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-de38a434-8af8-4055-a1e0-6177ae3f15c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:51.908639 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 707/1413] 10.222.0.15 () {58 vars in 1015 bytes} [Wed Jul 20 14:40:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:40:54.428559 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a557b94-aec6-4c3d-9c0d-6b33b51bb986 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:54.429169 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a557b94-aec6-4c3d-9c0d-6b33b51bb986 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:54.429169 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a557b94-aec6-4c3d-9c0d-6b33b51bb986 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:54.430500 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 14:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:40:54.437145 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45fb4c-17d6-481b-a4fa-edc8c11875e4 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:54.437587 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45fb4c-17d6-481b-a4fa-edc8c11875e4 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:54.437734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45fb4c-17d6-481b-a4fa-edc8c11875e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:54.518874 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-da45fb4c-17d6-481b-a4fa-edc8c11875e4 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:40:54.520180 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-da45fb4c-17d6-481b-a4fa-edc8c11875e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:54.578797 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:54.623510 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 14:40:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:40:54.670608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a557b94-aec6-4c3d-9c0d-6b33b51bb986 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:40:54.672988 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-6a557b94-aec6-4c3d-9c0d-6b33b51bb986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:54.829628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a557b94-aec6-4c3d-9c0d-6b33b51bb986 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:40:54.907539 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:54.949759 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1814b05-f99e-4248-8a3a-3562ef2804f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:40:54.950439 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1814b05-f99e-4248-8a3a-3562ef2804f5 service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:54.950726 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1814b05-f99e-4248-8a3a-3562ef2804f5 service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:54.951007 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1814b05-f99e-4248-8a3a-3562ef2804f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:54.952772 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-a1814b05-f99e-4248-8a3a-3562ef2804f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:55.050992 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1814b05-f99e-4248-8a3a-3562ef2804f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:40:55.053256 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1814b05-f99e-4248-8a3a-3562ef2804f5 service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:40:55.082013 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 708/1416] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:40:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:40:55.996911 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb4773-92e9-47b3-bdc7-deb1c363cdf2 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:55.997864 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb4773-92e9-47b3-bdc7-deb1c363cdf2 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:55.997864 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb4773-92e9-47b3-bdc7-deb1c363cdf2 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:56.000252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 14:40:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:40:56.006592 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43b60e-786a-4165-89fb-6d2f025d5965 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:56.007095 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43b60e-786a-4165-89fb-6d2f025d5965 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:56.007224 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43b60e-786a-4165-89fb-6d2f025d5965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:56.098972 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-5c43b60e-786a-4165-89fb-6d2f025d5965 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:40:56.100342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-5c43b60e-786a-4165-89fb-6d2f025d5965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:56.137311 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:56.210716 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 14:40:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:40:56.365695 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb4773-92e9-47b3-bdc7-deb1c363cdf2 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:40:56.367276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-09fb4773-92e9-47b3-bdc7-deb1c363cdf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:56.517964 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb4773-92e9-47b3-bdc7-deb1c363cdf2 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:40:56.564092 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:56.640646 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7307c-bfc1-4b4b-b3f2-4fb7a7e5df4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:40:56.641522 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7307c-bfc1-4b4b-b3f2-4fb7a7e5df4a service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:56.641942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7307c-bfc1-4b4b-b3f2-4fb7a7e5df4a service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:56.642368 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7307c-bfc1-4b4b-b3f2-4fb7a7e5df4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:56.645210 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-0db7307c-bfc1-4b4b-b3f2-4fb7a7e5df4a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:56.756637 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7307c-bfc1-4b4b-b3f2-4fb7a7e5df4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:40:56.758288 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7307c-bfc1-4b4b-b3f2-4fb7a7e5df4a service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:40:56.777423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 710/1419] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:40:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:40:57.660181 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438efe8-fa85-4a31-ab4f-f4fba0644e78 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:57.660765 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438efe8-fa85-4a31-ab4f-f4fba0644e78 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:57.660765 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438efe8-fa85-4a31-ab4f-f4fba0644e78 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:57.662480 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 14:40:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:40:57.667797 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0575c43-c9b4-4318-bada-99606afd55fe None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:57.668225 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0575c43-c9b4-4318-bada-99606afd55fe None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:57.668341 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0575c43-c9b4-4318-bada-99606afd55fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:57.725733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:57.752561 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-c0575c43-c9b4-4318-bada-99606afd55fe None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:40:57.753711 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-c0575c43-c9b4-4318-bada-99606afd55fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:57.867355 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 14:40:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:40:58.019204 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b438efe8-fa85-4a31-ab4f-f4fba0644e78 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:40:58.020946 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-b438efe8-fa85-4a31-ab4f-f4fba0644e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:58.175467 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b438efe8-fa85-4a31-ab4f-f4fba0644e78 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:40:58.188591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:58.298376 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f1cf7-30c5-451d-9a15-ef82589045b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:40:58.299040 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f1cf7-30c5-451d-9a15-ef82589045b5 service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:58.299348 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f1cf7-30c5-451d-9a15-ef82589045b5 service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:58.299639 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f1cf7-30c5-451d-9a15-ef82589045b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:58.301324 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-604f1cf7-30c5-451d-9a15-ef82589045b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:58.398139 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604f1cf7-30c5-451d-9a15-ef82589045b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:40:58.399789 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604f1cf7-30c5-451d-9a15-ef82589045b5 service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:40:58.418253 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 711/1422] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:40:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:40:59.296979 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc3a50-e6c4-4674-a72d-7e9a66177e33 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:59.297577 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc3a50-e6c4-4674-a72d-7e9a66177e33 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:59.297577 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc3a50-e6c4-4674-a72d-7e9a66177e33 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:59.298942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 14:40:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:40:59.305683 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e215763-a2e8-4cb2-a1be-6f8e512e2ceb None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:59.306410 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e215763-a2e8-4cb2-a1be-6f8e512e2ceb None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:59.306562 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e215763-a2e8-4cb2-a1be-6f8e512e2ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:59.335631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:59.393769 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-5e215763-a2e8-4cb2-a1be-6f8e512e2ceb None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:40:59.395028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-5e215763-a2e8-4cb2-a1be-6f8e512e2ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:59.499324 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 14:40:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:40:59.646851 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc3a50-e6c4-4674-a72d-7e9a66177e33 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:40:59.648347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-0ecc3a50-e6c4-4674-a72d-7e9a66177e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:40:59.766893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:59.804386 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc3a50-e6c4-4674-a72d-7e9a66177e33 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:40:59.926974 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a6549-51fb-4fc4-9d5e-c7f7538d957c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:40:59.927579 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a6549-51fb-4fc4-9d5e-c7f7538d957c service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:40:59.927867 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a6549-51fb-4fc4-9d5e-c7f7538d957c service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:40:59.928144 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a6549-51fb-4fc4-9d5e-c7f7538d957c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:40:59.929790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-c06a6549-51fb-4fc4-9d5e-c7f7538d957c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:00.010927 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:00.026201 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a6549-51fb-4fc4-9d5e-c7f7538d957c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:41:00.027846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a6549-51fb-4fc4-9d5e-c7f7538d957c service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:41:00.046625 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 713/1425] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:40:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:41:00.889962 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7e290-75c2-416b-987d-3c020ee7d1d5 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:00.890864 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7e290-75c2-416b-987d-3c020ee7d1d5 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:00.890864 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7e290-75c2-416b-987d-3c020ee7d1d5 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:00.893067 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 14:41:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:41:00.898190 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5ed27-c6b6-40ed-a7c0-bf2b9592c46f None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:00.898669 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5ed27-c6b6-40ed-a7c0-bf2b9592c46f None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:00.898775 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5ed27-c6b6-40ed-a7c0-bf2b9592c46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:01.000801 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-53a5ed27-c6b6-40ed-a7c0-bf2b9592c46f None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:41:01.002017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-53a5ed27-c6b6-40ed-a7c0-bf2b9592c46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:01.104872 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 14:41:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:41:01.256354 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f7e290-75c2-416b-987d-3c020ee7d1d5 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:41:01.258710 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-33f7e290-75c2-416b-987d-3c020ee7d1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:01.355063 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:01.421902 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f7e290-75c2-416b-987d-3c020ee7d1d5 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:41:01.548642 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e311137a-47bc-4857-97f8-2e8e92b1d69a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:41:01.549768 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311137a-47bc-4857-97f8-2e8e92b1d69a service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:01.550285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311137a-47bc-4857-97f8-2e8e92b1d69a service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:01.550851 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311137a-47bc-4857-97f8-2e8e92b1d69a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:01.553315 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e311137a-47bc-4857-97f8-2e8e92b1d69a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:01.616784 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:01.665632 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e311137a-47bc-4857-97f8-2e8e92b1d69a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:41:01.667349 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e311137a-47bc-4857-97f8-2e8e92b1d69a service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:41:01.686996 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 714/1428] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:41:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:41:02.488337 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775079d-c5b4-4a23-a22e-c42f67bf3e16 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:02.488737 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775079d-c5b4-4a23-a22e-c42f67bf3e16 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:02.488935 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775079d-c5b4-4a23-a22e-c42f67bf3e16 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:02.491526 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 14:41:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:41:02.497860 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470dd10-3222-4319-bc6c-9ec984545983 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:02.498355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470dd10-3222-4319-bc6c-9ec984545983 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:02.498482 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470dd10-3222-4319-bc6c-9ec984545983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:02.584245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-a470dd10-3222-4319-bc6c-9ec984545983 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:41:02.585505 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-a470dd10-3222-4319-bc6c-9ec984545983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:02.696270 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 14:41:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:41:02.856653 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775079d-c5b4-4a23-a22e-c42f67bf3e16 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:41:02.859112 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-d775079d-c5b4-4a23-a22e-c42f67bf3e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:02.919926 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:03.021851 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775079d-c5b4-4a23-a22e-c42f67bf3e16 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:41:03.141727 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0a9584-d79f-4f86-9916-7ca2261477ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:41:03.142295 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a9584-d79f-4f86-9916-7ca2261477ca service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:03.142592 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a9584-d79f-4f86-9916-7ca2261477ca service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:03.142853 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a9584-d79f-4f86-9916-7ca2261477ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:03.144586 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-cd0a9584-d79f-4f86-9916-7ca2261477ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:03.151709 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:03.234705 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0a9584-d79f-4f86-9916-7ca2261477ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:41:03.236321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0a9584-d79f-4f86-9916-7ca2261477ca service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:41:03.260751 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 716/1431] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:41:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:41:04.059602 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7c329-35f7-495a-94e3-d884edd617b0 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:04.060190 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7c329-35f7-495a-94e3-d884edd617b0 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:04.060190 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7c329-35f7-495a-94e3-d884edd617b0 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:04.062294 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 14:41:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:41:04.069943 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea9b8e-8090-44fd-9936-e4da8cb917fd None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:04.070360 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea9b8e-8090-44fd-9936-e4da8cb917fd None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:04.070521 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea9b8e-8090-44fd-9936-e4da8cb917fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:04.165883 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-b1ea9b8e-8090-44fd-9936-e4da8cb917fd None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:41:04.167141 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-b1ea9b8e-8090-44fd-9936-e4da8cb917fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:04.241715 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:04.272612 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 14:41:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:41:04.422326 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7c329-35f7-495a-94e3-d884edd617b0 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:41:04.423826 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-c8c7c329-35f7-495a-94e3-d884edd617b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:04.446270 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:04.588519 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7c329-35f7-495a-94e3-d884edd617b0 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:41:04.681716 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:04.711717 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3420b7aa-9079-4918-bd82-c18c8b2c7b13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:41:04.712368 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420b7aa-9079-4918-bd82-c18c8b2c7b13 service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:04.712650 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420b7aa-9079-4918-bd82-c18c8b2c7b13 service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:04.712921 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420b7aa-9079-4918-bd82-c18c8b2c7b13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:04.714590 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-3420b7aa-9079-4918-bd82-c18c8b2c7b13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:04.824634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3420b7aa-9079-4918-bd82-c18c8b2c7b13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:41:04.826296 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3420b7aa-9079-4918-bd82-c18c8b2c7b13 service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:41:04.844515 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 717/1434] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:41:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:41:05.652917 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34eedf-5ee3-4000-b452-8bcdacf39478 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:05.653818 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34eedf-5ee3-4000-b452-8bcdacf39478 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:05.653818 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34eedf-5ee3-4000-b452-8bcdacf39478 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:05.656143 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 14:41:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:41:05.661654 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93ab30-25c1-4693-92a5-34a47947c14d None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:05.662134 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93ab30-25c1-4693-92a5-34a47947c14d None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:05.662547 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93ab30-25c1-4693-92a5-34a47947c14d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:05.743007 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-3d93ab30-25c1-4693-92a5-34a47947c14d None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:41:05.744269 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-3d93ab30-25c1-4693-92a5-34a47947c14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:05.786639 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:05.847311 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 14:41:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:41:06.009451 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34eedf-5ee3-4000-b452-8bcdacf39478 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:41:06.011819 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-6c34eedf-5ee3-4000-b452-8bcdacf39478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:06.180536 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34eedf-5ee3-4000-b452-8bcdacf39478 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:41:06.241278 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:06.315566 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557e6c4-9a4b-4ce3-8b09-70b494c60078 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:41:06.316225 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557e6c4-9a4b-4ce3-8b09-70b494c60078 service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:06.316610 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557e6c4-9a4b-4ce3-8b09-70b494c60078 service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:06.316894 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557e6c4-9a4b-4ce3-8b09-70b494c60078 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:06.318542 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-4557e6c4-9a4b-4ce3-8b09-70b494c60078 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:06.414507 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557e6c4-9a4b-4ce3-8b09-70b494c60078 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:41:06.416152 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557e6c4-9a4b-4ce3-8b09-70b494c60078 service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:41:06.435477 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 719/1437] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:41:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:41:07.248014 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e7ed5-3ec8-4361-ba01-450c098d8e63 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:07.248733 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e7ed5-3ec8-4361-ba01-450c098d8e63 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:07.248733 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e7ed5-3ec8-4361-ba01-450c098d8e63 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:07.250824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 14:41:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:41:07.256353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52c0b7-7385-4a43-a115-0e3dab9e480c None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:07.256817 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52c0b7-7385-4a43-a115-0e3dab9e480c None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:07.256931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52c0b7-7385-4a43-a115-0e3dab9e480c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:07.330562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-bd52c0b7-7385-4a43-a115-0e3dab9e480c None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:41:07.331789 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-bd52c0b7-7385-4a43-a115-0e3dab9e480c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:07.333970 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:07.431452 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 14:41:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:41:07.583848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e7ed5-3ec8-4361-ba01-450c098d8e63 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:41:07.585423 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-1f6e7ed5-3ec8-4361-ba01-450c098d8e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:07.758378 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e7ed5-3ec8-4361-ba01-450c098d8e63 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:41:07.785448 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:07.874721 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b463d22-83c6-47e6-9229-3b12d998b40b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:41:07.875329 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b463d22-83c6-47e6-9229-3b12d998b40b service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:07.875640 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b463d22-83c6-47e6-9229-3b12d998b40b service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:07.875921 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b463d22-83c6-47e6-9229-3b12d998b40b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:07.877662 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-0b463d22-83c6-47e6-9229-3b12d998b40b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:07.979832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b463d22-83c6-47e6-9229-3b12d998b40b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:41:07.981462 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b463d22-83c6-47e6-9229-3b12d998b40b service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:41:08.000026 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 720/1440] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:41:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:41:09.933484 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dff256-da2d-4ea0-a26d-cf538086774b None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:09.934077 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dff256-da2d-4ea0-a26d-cf538086774b None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:09.934077 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dff256-da2d-4ea0-a26d-cf538086774b None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:09.936266 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 14:41:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:41:09.941888 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b2286-cdcb-4423-9606-2024ef1783e2 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:09.942357 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b2286-cdcb-4423-9606-2024ef1783e2 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:09.942470 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b2286-cdcb-4423-9606-2024ef1783e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:09.984499 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:10.010560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-5f7b2286-cdcb-4423-9606-2024ef1783e2 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:41:10.011795 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-5f7b2286-cdcb-4423-9606-2024ef1783e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:10.125314 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 14:41:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:41:10.165413 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dff256-da2d-4ea0-a26d-cf538086774b None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:41:10.166978 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-b8dff256-da2d-4ea0-a26d-cf538086774b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:10.272353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dff256-da2d-4ea0-a26d-cf538086774b None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:41:10.307200 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:10.352980 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dffbbf0-0794-442c-9ccb-8c5c15158c20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:41:10.353642 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffbbf0-0794-442c-9ccb-8c5c15158c20 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:10.353930 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffbbf0-0794-442c-9ccb-8c5c15158c20 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:10.354208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffbbf0-0794-442c-9ccb-8c5c15158c20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:10.355911 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-3dffbbf0-0794-442c-9ccb-8c5c15158c20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:10.477004 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dffbbf0-0794-442c-9ccb-8c5c15158c20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:41:10.478453 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dffbbf0-0794-442c-9ccb-8c5c15158c20 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:41:10.498804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 722/1443] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:41:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:41:12.746130 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea2bbc-3153-46f0-8142-b8c1ee26f12a None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:12.746657 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea2bbc-3153-46f0-8142-b8c1ee26f12a None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:12.746657 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea2bbc-3153-46f0-8142-b8c1ee26f12a None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:12.748858 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 14:41:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:41:12.757160 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda2832-5d2a-46b6-bd26-b03168089d71 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:12.757590 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda2832-5d2a-46b6-bd26-b03168089d71 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:12.757769 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda2832-5d2a-46b6-bd26-b03168089d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:12.830936 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-5dda2832-5d2a-46b6-bd26-b03168089d71 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:41:12.832268 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-5dda2832-5d2a-46b6-bd26-b03168089d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:12.845461 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:12.935459 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 14:41:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:41:12.973818 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea2bbc-3153-46f0-8142-b8c1ee26f12a None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:41:12.975452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e3ea2bbc-3153-46f0-8142-b8c1ee26f12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:13.081854 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea2bbc-3153-46f0-8142-b8c1ee26f12a None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:41:13.094299 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:13.170528 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2212850-10c9-4bdc-bd33-67b66306d642 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:41:13.171241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2212850-10c9-4bdc-bd33-67b66306d642 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:13.171528 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2212850-10c9-4bdc-bd33-67b66306d642 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:13.171805 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2212850-10c9-4bdc-bd33-67b66306d642 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:13.173481 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-a2212850-10c9-4bdc-bd33-67b66306d642 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:13.302922 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2212850-10c9-4bdc-bd33-67b66306d642 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:41:13.304484 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2212850-10c9-4bdc-bd33-67b66306d642 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:41:13.323739 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 723/1446] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:41:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:41:15.695128 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4586ea-4027-448a-967a-093d75266304 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:15.696067 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4586ea-4027-448a-967a-093d75266304 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:15.696067 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4586ea-4027-448a-967a-093d75266304 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:15.698311 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 14:41:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:41:15.706489 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470f446-8999-4fcd-8610-bcfe5026d220 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:15.706937 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470f446-8999-4fcd-8610-bcfe5026d220 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:15.707125 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470f446-8999-4fcd-8610-bcfe5026d220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:15.755162 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:15.773653 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-1470f446-8999-4fcd-8610-bcfe5026d220 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:41:15.774882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-1470f446-8999-4fcd-8610-bcfe5026d220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:15.880792 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 14:41:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:41:15.924639 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4586ea-4027-448a-967a-093d75266304 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:41:15.926788 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-1f4586ea-4027-448a-967a-093d75266304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:16.046043 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4586ea-4027-448a-967a-093d75266304 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:41:16.089028 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:16.126458 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86330b-1762-4a49-9cd5-035953d533dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:41:16.127160 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86330b-1762-4a49-9cd5-035953d533dc service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:16.127450 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86330b-1762-4a49-9cd5-035953d533dc service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:16.127726 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86330b-1762-4a49-9cd5-035953d533dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:16.129498 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-5b86330b-1762-4a49-9cd5-035953d533dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:16.255399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b86330b-1762-4a49-9cd5-035953d533dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:41:16.256862 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b86330b-1762-4a49-9cd5-035953d533dc service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:41:16.275662 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 725/1449] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:41:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:41:18.401723 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74c576-df6a-44c0-93c3-4c942a6f84d8 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:18.402384 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74c576-df6a-44c0-93c3-4c942a6f84d8 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:18.402384 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74c576-df6a-44c0-93c3-4c942a6f84d8 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:18.403792 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 14:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:41:18.410626 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f391bf-6e70-42f6-be01-cc94ad09bc9a None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:18.411113 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f391bf-6e70-42f6-be01-cc94ad09bc9a None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:18.411242 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f391bf-6e70-42f6-be01-cc94ad09bc9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:18.483464 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-e0f391bf-6e70-42f6-be01-cc94ad09bc9a None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:41:18.484745 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-e0f391bf-6e70-42f6-be01-cc94ad09bc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:18.506998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:18.589338 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 14:41:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:41:18.632863 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74c576-df6a-44c0-93c3-4c942a6f84d8 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:41:18.634487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-3c74c576-df6a-44c0-93c3-4c942a6f84d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:18.743410 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74c576-df6a-44c0-93c3-4c942a6f84d8 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:41:18.766987 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:18.821676 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9bd2d6-9634-42e6-b0c9-254a45d12d7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:41:18.822357 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9bd2d6-9634-42e6-b0c9-254a45d12d7b service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:18.822640 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9bd2d6-9634-42e6-b0c9-254a45d12d7b service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:18.822906 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9bd2d6-9634-42e6-b0c9-254a45d12d7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:18.824641 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-fa9bd2d6-9634-42e6-b0c9-254a45d12d7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:18.935832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9bd2d6-9634-42e6-b0c9-254a45d12d7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:41:18.937258 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9bd2d6-9634-42e6-b0c9-254a45d12d7b service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:41:18.955561 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 726/1452] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:41:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:41:21.109768 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22017a86-3f5e-4a53-8da5-2e88f6023db6 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:21.110693 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22017a86-3f5e-4a53-8da5-2e88f6023db6 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:21.110693 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22017a86-3f5e-4a53-8da5-2e88f6023db6 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:21.112956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 14:41:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:41:21.119395 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb8302-882f-41b0-abd7-2e0c0ab4b9c4 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:21.119872 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb8302-882f-41b0-abd7-2e0c0ab4b9c4 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:21.120037 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb8302-882f-41b0-abd7-2e0c0ab4b9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:21.186201 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:21.189411 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-96cb8302-882f-41b0-abd7-2e0c0ab4b9c4 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:41:21.191526 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-96cb8302-882f-41b0-abd7-2e0c0ab4b9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:21.296016 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 14:41:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:41:21.336537 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22017a86-3f5e-4a53-8da5-2e88f6023db6 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:41:21.338920 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-22017a86-3f5e-4a53-8da5-2e88f6023db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:21.465779 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22017a86-3f5e-4a53-8da5-2e88f6023db6 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:41:21.519803 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:21.548695 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ca5ef-ce68-4752-b74c-db7564d83d25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:41:21.549381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ca5ef-ce68-4752-b74c-db7564d83d25 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:21.549662 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ca5ef-ce68-4752-b74c-db7564d83d25 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:21.549933 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ca5ef-ce68-4752-b74c-db7564d83d25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:21.551716 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-c87ca5ef-ce68-4752-b74c-db7564d83d25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:21.810009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ca5ef-ce68-4752-b74c-db7564d83d25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:41:21.811624 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ca5ef-ce68-4752-b74c-db7564d83d25 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:41:21.831665 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 728/1455] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:41:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:41:23.956839 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996316ab-992c-4f3f-9d68-f57bbca0e37e None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:23.957444 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996316ab-992c-4f3f-9d68-f57bbca0e37e None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:23.957444 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996316ab-992c-4f3f-9d68-f57bbca0e37e None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:23.959088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 14:41:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:41:23.966847 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11003fd4-66d2-4aad-8180-abac84c1d014 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:23.967314 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11003fd4-66d2-4aad-8180-abac84c1d014 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:23.967438 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11003fd4-66d2-4aad-8180-abac84c1d014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:24.038566 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-11003fd4-66d2-4aad-8180-abac84c1d014 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:41:24.039756 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-11003fd4-66d2-4aad-8180-abac84c1d014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:24.070654 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:24.154278 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 14:41:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:41:24.198342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996316ab-992c-4f3f-9d68-f57bbca0e37e None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:41:24.200732 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-996316ab-992c-4f3f-9d68-f57bbca0e37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:24.318784 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996316ab-992c-4f3f-9d68-f57bbca0e37e None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:41:24.348821 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:24.395738 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5035f112-6556-4fb8-b6e9-2cbb03a2d4de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:41:24.396419 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035f112-6556-4fb8-b6e9-2cbb03a2d4de service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:24.396705 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035f112-6556-4fb8-b6e9-2cbb03a2d4de service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:24.396984 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035f112-6556-4fb8-b6e9-2cbb03a2d4de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:24.398689 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-5035f112-6556-4fb8-b6e9-2cbb03a2d4de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:24.521066 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5035f112-6556-4fb8-b6e9-2cbb03a2d4de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:41:24.522849 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5035f112-6556-4fb8-b6e9-2cbb03a2d4de service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:41:24.543545 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 729/1458] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:41:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:41:26.641520 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be402fe2-d98a-4bbd-8ea0-d2c8278c56bb None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:26.642420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be402fe2-d98a-4bbd-8ea0-d2c8278c56bb None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:26.642420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be402fe2-d98a-4bbd-8ea0-d2c8278c56bb None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:26.644712 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 14:41:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:41:26.653461 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a2738-cd12-4c12-97b3-e628d256f242 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:26.653893 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a2738-cd12-4c12-97b3-e628d256f242 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:26.654034 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a2738-cd12-4c12-97b3-e628d256f242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:26.731346 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-057a2738-cd12-4c12-97b3-e628d256f242 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:41:26.732569 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-057a2738-cd12-4c12-97b3-e628d256f242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:26.741552 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:26.836686 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 14:41:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:41:26.880238 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be402fe2-d98a-4bbd-8ea0-d2c8278c56bb None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:41:26.881965 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-be402fe2-d98a-4bbd-8ea0-d2c8278c56bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:26.994384 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be402fe2-d98a-4bbd-8ea0-d2c8278c56bb None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:41:27.053541 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:27.073584 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8be695-74b4-457a-a812-77a7cf4a348e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:41:27.074247 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8be695-74b4-457a-a812-77a7cf4a348e service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:27.074520 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8be695-74b4-457a-a812-77a7cf4a348e service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:27.074797 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8be695-74b4-457a-a812-77a7cf4a348e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:27.076479 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-6a8be695-74b4-457a-a812-77a7cf4a348e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:27.191078 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8be695-74b4-457a-a812-77a7cf4a348e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:41:27.192562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8be695-74b4-457a-a812-77a7cf4a348e service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:41:27.212636 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 731/1461] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:41:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:41:29.299820 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d173488-b74a-43fe-88a6-4a6530fb31cb None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:29.300412 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d173488-b74a-43fe-88a6-4a6530fb31cb None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:29.300412 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d173488-b74a-43fe-88a6-4a6530fb31cb None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:29.301781 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 14:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:41:29.308364 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5cedd-db1f-4fbd-b9da-dcc840c9f66e None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:29.308794 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5cedd-db1f-4fbd-b9da-dcc840c9f66e None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:29.309025 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5cedd-db1f-4fbd-b9da-dcc840c9f66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:29.377387 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-c6a5cedd-db1f-4fbd-b9da-dcc840c9f66e None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:41:29.378508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-c6a5cedd-db1f-4fbd-b9da-dcc840c9f66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:29.418118 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:29.484592 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 14:41:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:41:29.525804 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d173488-b74a-43fe-88a6-4a6530fb31cb None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:41:29.528849 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-1d173488-b74a-43fe-88a6-4a6530fb31cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:29.649430 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d173488-b74a-43fe-88a6-4a6530fb31cb None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:41:29.696918 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:29.741218 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a34447c-a3d3-43e6-8ec1-b96341962973 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:41:29.741916 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34447c-a3d3-43e6-8ec1-b96341962973 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:29.742205 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34447c-a3d3-43e6-8ec1-b96341962973 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:29.742480 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34447c-a3d3-43e6-8ec1-b96341962973 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:29.744458 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-8a34447c-a3d3-43e6-8ec1-b96341962973 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:29.864000 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a34447c-a3d3-43e6-8ec1-b96341962973 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:41:29.865478 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a34447c-a3d3-43e6-8ec1-b96341962973 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:41:29.883999 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 732/1464] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:41:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:41:32.001842 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487bc50-c80c-4aac-bc68-8040e306bd02 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:32.002434 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487bc50-c80c-4aac-bc68-8040e306bd02 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:32.002434 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487bc50-c80c-4aac-bc68-8040e306bd02 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:32.003774 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 14:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:41:32.009495 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70cdf9-4512-4985-8da1-5a8f1cbd6596 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:32.009975 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70cdf9-4512-4985-8da1-5a8f1cbd6596 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:32.010084 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70cdf9-4512-4985-8da1-5a8f1cbd6596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:32.086889 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-cf70cdf9-4512-4985-8da1-5a8f1cbd6596 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:41:32.088144 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-cf70cdf9-4512-4985-8da1-5a8f1cbd6596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:32.107815 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:32.192483 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 14:41:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:41:32.230190 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487bc50-c80c-4aac-bc68-8040e306bd02 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:41:32.232661 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-e487bc50-c80c-4aac-bc68-8040e306bd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:32.350091 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487bc50-c80c-4aac-bc68-8040e306bd02 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:41:32.424708 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:32.432575 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797fb678-70b3-4772-bee1-9fd48eb58e8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:41:32.433230 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fb678-70b3-4772-bee1-9fd48eb58e8a service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:32.433509 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fb678-70b3-4772-bee1-9fd48eb58e8a service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:32.433781 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fb678-70b3-4772-bee1-9fd48eb58e8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:32.435465 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-797fb678-70b3-4772-bee1-9fd48eb58e8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:32.566040 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797fb678-70b3-4772-bee1-9fd48eb58e8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:41:32.567536 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797fb678-70b3-4772-bee1-9fd48eb58e8a service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:41:32.588872 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 734/1467] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:41:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:41:34.606447 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6bcc2-66b8-4eb6-8079-5bfe85ecec5f None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:34.607079 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6bcc2-66b8-4eb6-8079-5bfe85ecec5f None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:34.607079 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6bcc2-66b8-4eb6-8079-5bfe85ecec5f None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:34.608402 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 14:41:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:41:34.613870 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec2d65-6867-446b-8e9e-7f1dd939e7d8 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:34.614305 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec2d65-6867-446b-8e9e-7f1dd939e7d8 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:34.614412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec2d65-6867-446b-8e9e-7f1dd939e7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:34.683874 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-55ec2d65-6867-446b-8e9e-7f1dd939e7d8 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:41:34.685089 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-55ec2d65-6867-446b-8e9e-7f1dd939e7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:34.739139 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:34.799030 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 14:41:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:41:34.835959 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6bcc2-66b8-4eb6-8079-5bfe85ecec5f None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:41:34.837548 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-c9d6bcc2-66b8-4eb6-8079-5bfe85ecec5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:34.955799 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6bcc2-66b8-4eb6-8079-5bfe85ecec5f None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:41:35.005897 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:35.037837 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d010c-5497-4a42-b573-2cfc13b0bad5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:41:35.038675 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d010c-5497-4a42-b573-2cfc13b0bad5 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:35.038967 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d010c-5497-4a42-b573-2cfc13b0bad5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:35.039312 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d010c-5497-4a42-b573-2cfc13b0bad5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:35.041021 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-146d010c-5497-4a42-b573-2cfc13b0bad5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:35.157248 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d010c-5497-4a42-b573-2cfc13b0bad5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:41:35.158704 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d010c-5497-4a42-b573-2cfc13b0bad5 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:41:35.177437 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 735/1470] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:41:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:41:44.796034 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=22 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:45.175103 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=66 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:54.824286 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ff0b8-557b-49e9-9231-bdd5b3423bd7 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:54.825160 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ff0b8-557b-49e9-9231-bdd5b3423bd7 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:54.825160 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ff0b8-557b-49e9-9231-bdd5b3423bd7 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:54.827414 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 736/1471] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:41:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:41:54.833940 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd42f35-56b3-43bb-9f6d-880fd344c520 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:54.834408 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd42f35-56b3-43bb-9f6d-880fd344c520 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:54.834522 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd42f35-56b3-43bb-9f6d-880fd344c520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:54.900593 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-2bd42f35-56b3-43bb-9f6d-880fd344c520 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:41:54.901807 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-2bd42f35-56b3-43bb-9f6d-880fd344c520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:55.015178 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 736/1472] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:41:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:41:55.100619 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21a230-7dfb-40b7-aeb7-9f3505efc02d None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:55.100906 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21a230-7dfb-40b7-aeb7-9f3505efc02d None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:55.101181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21a230-7dfb-40b7-aeb7-9f3505efc02d None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:55.102632 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 737/1473] 10.222.0.15 () {54 vars in 966 bytes} [Wed Jul 20 14:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:41:55.108544 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10debe56-d052-4dda-9838-c285bbfbc40e None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:55.108922 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10debe56-d052-4dda-9838-c285bbfbc40e None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:55.109225 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10debe56-d052-4dda-9838-c285bbfbc40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:55.192487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-10debe56-d052-4dda-9838-c285bbfbc40e None None] MFA Rules not processed for user `15d7d59080cd4832a45dbb6c0349cf4c`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:41:55.193711 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-10debe56-d052-4dda-9838-c285bbfbc40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:55.214917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:55.275229 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 737/1474] 10.222.0.15 () {58 vars in 1063 bytes} [Wed Jul 20 14:41:55 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:41:55.282478 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a21a230-7dfb-40b7-aeb7-9f3505efc02d None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:41:55.284178 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-3a21a230-7dfb-40b7-aeb7-9f3505efc02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:55.393039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a21a230-7dfb-40b7-aeb7-9f3505efc02d None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:41:55.467951 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5049f-88b3-4145-ab32-9bb95262dcaa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d7d59080cd4832a45dbb6c0349cf4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:41:55.468600 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5049f-88b3-4145-ab32-9bb95262dcaa service glance] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:55.468890 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5049f-88b3-4145-ab32-9bb95262dcaa service glance] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:55.469163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5049f-88b3-4145-ab32-9bb95262dcaa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:55.470808 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-dca5049f-88b3-4145-ab32-9bb95262dcaa service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:55.531644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:55.590281 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5049f-88b3-4145-ab32-9bb95262dcaa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:41:55.591760 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5049f-88b3-4145-ab32-9bb95262dcaa service glance] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:41:55.611426 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 738/1475] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:41:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:41:57.706234 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0e6de-82e3-413d-b172-e72d5ddd7c36 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:57.707118 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0e6de-82e3-413d-b172-e72d5ddd7c36 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:57.707118 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0e6de-82e3-413d-b172-e72d5ddd7c36 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:57.709524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 738/1476] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:41:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:41:57.714976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd6843-4ad3-44b4-a51d-5be5f70026a9 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:57.715468 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd6843-4ad3-44b4-a51d-5be5f70026a9 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:57.715574 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd6843-4ad3-44b4-a51d-5be5f70026a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:57.782697 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-cedd6843-4ad3-44b4-a51d-5be5f70026a9 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:41:57.783966 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-cedd6843-4ad3-44b4-a51d-5be5f70026a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:57.893507 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 739/1477] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:41:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:41:57.917468 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0e6de-82e3-413d-b172-e72d5ddd7c36 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:41:57.919016 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-c5f0e6de-82e3-413d-b172-e72d5ddd7c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:41:58.060956 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0e6de-82e3-413d-b172-e72d5ddd7c36 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:41:58.110731 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:41:58.156155 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7403277-b93f-4cd6-ab04-548a2e1a8b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:41:58.156861 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7403277-b93f-4cd6-ab04-548a2e1a8b08 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:41:58.157159 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7403277-b93f-4cd6-ab04-548a2e1a8b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:41:58.157441 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7403277-b93f-4cd6-ab04-548a2e1a8b08 admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:41:58.158211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7403277-b93f-4cd6-ab04-548a2e1a8b08 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:41:58.160044 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: WARNING py.warnings [None req-c7403277-b93f-4cd6-ab04-548a2e1a8b08 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 14:41:58.160044 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: warnings.warn(msg) Jul 20 14:41:58.160044 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: Jul 20 14:41:58.161290 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7403277-b93f-4cd6-ab04-548a2e1a8b08 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:41:58.183696 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 739/1478] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jul 20 14:41:57 2022] GET /identity/v3/projects => generated 2553 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:42:00.201217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a10b0-8491-4d10-8244-1c22e26ea515 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:42:00.202154 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a10b0-8491-4d10-8244-1c22e26ea515 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:42:00.202154 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a10b0-8491-4d10-8244-1c22e26ea515 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:42:00.204389 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 740/1479] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:42:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:42:00.210906 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a99a1d-2777-4565-8eac-890c0522a191 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:42:00.211672 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a99a1d-2777-4565-8eac-890c0522a191 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:42:00.211878 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a99a1d-2777-4565-8eac-890c0522a191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:42:00.285440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-18a99a1d-2777-4565-8eac-890c0522a191 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:42:00.286659 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-18a99a1d-2777-4565-8eac-890c0522a191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:42:00.386572 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 740/1480] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:42:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:42:00.436797 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a10b0-8491-4d10-8244-1c22e26ea515 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:42:00.438397 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-f80a10b0-8491-4d10-8244-1c22e26ea515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:42:00.448824 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:42:00.594595 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a10b0-8491-4d10-8244-1c22e26ea515 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:42:00.680886 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:42:00.717136 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dabac0-abd5-4216-94f6-07f2194404e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:42:00.717818 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dabac0-abd5-4216-94f6-07f2194404e1 service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:42:00.718098 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dabac0-abd5-4216-94f6-07f2194404e1 service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:42:00.718367 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dabac0-abd5-4216-94f6-07f2194404e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:42:00.720054 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-f8dabac0-abd5-4216-94f6-07f2194404e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:42:00.811221 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dabac0-abd5-4216-94f6-07f2194404e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:42:00.812812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dabac0-abd5-4216-94f6-07f2194404e1 service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:42:00.830946 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 741/1481] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:42:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:42:03.066698 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37c201-9b11-4664-a8bd-c11125cfcb10 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:42:03.067393 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37c201-9b11-4664-a8bd-c11125cfcb10 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:42:03.067393 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37c201-9b11-4664-a8bd-c11125cfcb10 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:42:03.068714 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 741/1482] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:42:03.074401 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a358b-1247-4b2e-bbf5-716c741042cd None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:42:03.075269 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a358b-1247-4b2e-bbf5-716c741042cd None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:42:03.075716 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a358b-1247-4b2e-bbf5-716c741042cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:42:03.143867 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-658a358b-1247-4b2e-bbf5-716c741042cd None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:42:03.145138 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-658a358b-1247-4b2e-bbf5-716c741042cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:42:03.191790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:42:03.245878 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 742/1483] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:42:03.285805 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37c201-9b11-4664-a8bd-c11125cfcb10 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:42:03.287473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-6f37c201-9b11-4664-a8bd-c11125cfcb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:42:03.313129 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:42:03.411142 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37c201-9b11-4664-a8bd-c11125cfcb10 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:42:03.494088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea6eb8-3656-4faa-b666-4d37a1d5234c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:42:03.494765 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea6eb8-3656-4faa-b666-4d37a1d5234c service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:42:03.495076 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea6eb8-3656-4faa-b666-4d37a1d5234c service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:42:03.495369 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea6eb8-3656-4faa-b666-4d37a1d5234c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:42:03.497054 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-77ea6eb8-3656-4faa-b666-4d37a1d5234c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:42:03.575335 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:42:03.610657 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea6eb8-3656-4faa-b666-4d37a1d5234c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:42:03.612128 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea6eb8-3656-4faa-b666-4d37a1d5234c service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:42:03.630495 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 742/1484] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:42:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:42:05.705502 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77ecc6-153a-45ab-b9e3-187d8f8e8f00 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:42:05.706363 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77ecc6-153a-45ab-b9e3-187d8f8e8f00 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:42:05.706363 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77ecc6-153a-45ab-b9e3-187d8f8e8f00 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:42:05.708671 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 743/1485] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:42:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:42:05.714152 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4c07d-e377-4f3e-bda5-b0a8428bf056 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:42:05.714608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4c07d-e377-4f3e-bda5-b0a8428bf056 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:42:05.714734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4c07d-e377-4f3e-bda5-b0a8428bf056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:42:05.781073 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-e4a4c07d-e377-4f3e-bda5-b0a8428bf056 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:42:05.782289 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e4a4c07d-e377-4f3e-bda5-b0a8428bf056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:42:05.880720 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 743/1486] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:42:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:42:05.923480 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77ecc6-153a-45ab-b9e3-187d8f8e8f00 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:42:05.925950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-bf77ecc6-153a-45ab-b9e3-187d8f8e8f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:42:06.042999 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77ecc6-153a-45ab-b9e3-187d8f8e8f00 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:42:06.121297 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:42:06.123178 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36936527-3a37-4fa2-be2a-fc933ac6c2f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:42:06.123862 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36936527-3a37-4fa2-be2a-fc933ac6c2f9 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:42:06.124161 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36936527-3a37-4fa2-be2a-fc933ac6c2f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:42:06.124442 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36936527-3a37-4fa2-be2a-fc933ac6c2f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:42:06.126705 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-36936527-3a37-4fa2-be2a-fc933ac6c2f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:42:06.241019 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36936527-3a37-4fa2-be2a-fc933ac6c2f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:42:06.242466 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36936527-3a37-4fa2-be2a-fc933ac6c2f9 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:42:06.266015 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 744/1487] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:42:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:42:08.494080 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ebb8c-3e54-4b74-882e-cf151eba238f None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:42:08.494592 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ebb8c-3e54-4b74-882e-cf151eba238f None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:42:08.494592 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ebb8c-3e54-4b74-882e-cf151eba238f None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:42:08.496110 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 744/1488] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:42:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:42:08.501579 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13ed02-67aa-4c60-8baa-fc648755989d None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:42:08.502032 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13ed02-67aa-4c60-8baa-fc648755989d None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:42:08.502140 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13ed02-67aa-4c60-8baa-fc648755989d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:42:08.568092 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-bd13ed02-67aa-4c60-8baa-fc648755989d None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:42:08.569302 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-bd13ed02-67aa-4c60-8baa-fc648755989d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:42:08.622922 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:42:08.670185 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 745/1489] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:42:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:42:08.714894 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ebb8c-3e54-4b74-882e-cf151eba238f None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:42:08.717412 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-7d8ebb8c-3e54-4b74-882e-cf151eba238f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:42:08.760843 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:42:08.831027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ebb8c-3e54-4b74-882e-cf151eba238f None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:42:08.905044 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd13bb7-b67e-4ca7-8452-dd9e9c52e87b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:42:08.905705 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd13bb7-b67e-4ca7-8452-dd9e9c52e87b service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:42:08.905990 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd13bb7-b67e-4ca7-8452-dd9e9c52e87b service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:42:08.906264 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd13bb7-b67e-4ca7-8452-dd9e9c52e87b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:42:08.907984 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-1bd13bb7-b67e-4ca7-8452-dd9e9c52e87b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:42:08.990079 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:42:09.021801 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd13bb7-b67e-4ca7-8452-dd9e9c52e87b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:42:09.023210 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd13bb7-b67e-4ca7-8452-dd9e9c52e87b service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:42:09.046732 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 745/1490] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:42:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:42:11.331957 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf25bc-401b-4505-aa2d-8256eb64d5e4 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:42:11.332524 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf25bc-401b-4505-aa2d-8256eb64d5e4 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:42:11.332524 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf25bc-401b-4505-aa2d-8256eb64d5e4 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:42:11.333924 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 746/1491] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jul 20 14:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:42:11.340379 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae16057-d664-40be-b767-2f2073c4d62f None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:42:11.341088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae16057-d664-40be-b767-2f2073c4d62f None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:42:11.341286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae16057-d664-40be-b767-2f2073c4d62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:42:11.413508 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-5ae16057-d664-40be-b767-2f2073c4d62f None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:42:11.414730 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-5ae16057-d664-40be-b767-2f2073c4d62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:42:11.517472 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 746/1492] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jul 20 14:42:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:42:11.556651 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf25bc-401b-4505-aa2d-8256eb64d5e4 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:42:11.558181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-f9bf25bc-401b-4505-aa2d-8256eb64d5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:42:11.673475 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf25bc-401b-4505-aa2d-8256eb64d5e4 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:42:11.747739 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9e5d5-f0f9-448e-aa22-1bfd21dba9ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:42:11.748393 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9e5d5-f0f9-448e-aa22-1bfd21dba9ba service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:42:11.748693 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9e5d5-f0f9-448e-aa22-1bfd21dba9ba service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:42:11.748966 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9e5d5-f0f9-448e-aa22-1bfd21dba9ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:42:11.750600 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-fcc9e5d5-f0f9-448e-aa22-1bfd21dba9ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:42:11.760241 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:42:11.867363 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc9e5d5-f0f9-448e-aa22-1bfd21dba9ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:42:11.868871 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc9e5d5-f0f9-448e-aa22-1bfd21dba9ba service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:42:11.887641 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 747/1493] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:42:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:42:21.515682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=82 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:42:21.884806 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=50 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:43:46.608885 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850bed03-af36-48f2-8557-0edce53db316 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:43:46.610175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850bed03-af36-48f2-8557-0edce53db316 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:43:46.610175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850bed03-af36-48f2-8557-0edce53db316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:43:46.708837 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-850bed03-af36-48f2-8557-0edce53db316 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:43:46.710075 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-850bed03-af36-48f2-8557-0edce53db316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:43:46.833239 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 747/1494] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:43:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:43:46.842286 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aafa05-4231-4a9e-b975-d1bb128b9cdf None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:43:46.842827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aafa05-4231-4a9e-b975-d1bb128b9cdf None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:43:46.842827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aafa05-4231-4a9e-b975-d1bb128b9cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:43:46.928634 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-25aafa05-4231-4a9e-b975-d1bb128b9cdf None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:43:46.929864 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-25aafa05-4231-4a9e-b975-d1bb128b9cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:43:47.051477 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 748/1495] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:43:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:43:47.057241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850bed03-af36-48f2-8557-0edce53db316 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:43:47.058613 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-850bed03-af36-48f2-8557-0edce53db316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:43:47.071238 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850bed03-af36-48f2-8557-0edce53db316 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:43:47.148960 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:43:47.190726 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0c466-13bd-4c5c-a1f1-cc4c6585609a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:43:47.191582 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0c466-13bd-4c5c-a1f1-cc4c6585609a admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:43:47.191929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0c466-13bd-4c5c-a1f1-cc4c6585609a admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:43:47.192270 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0c466-13bd-4c5c-a1f1-cc4c6585609a admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:43:47.193159 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff0c466-13bd-4c5c-a1f1-cc4c6585609a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:43:47.193550 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff0c466-13bd-4c5c-a1f1-cc4c6585609a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:43:47.194777 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: WARNING py.warnings [None req-aff0c466-13bd-4c5c-a1f1-cc4c6585609a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 14:43:47.194777 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: warnings.warn(msg) Jul 20 14:43:47.194777 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: Jul 20 14:43:47.195933 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff0c466-13bd-4c5c-a1f1-cc4c6585609a admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:43:47.209837 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 748/1496] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:43:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:43:47.220483 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aafa05-4231-4a9e-b975-d1bb128b9cdf None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:43:47.236279 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aafa05-4231-4a9e-b975-d1bb128b9cdf None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:43:47.313872 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:43:47.358334 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be8558-6458-4f7a-9265-f035a6f77c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:43:47.359041 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be8558-6458-4f7a-9265-f035a6f77c68 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:43:47.359351 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be8558-6458-4f7a-9265-f035a6f77c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:43:47.359621 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be8558-6458-4f7a-9265-f035a6f77c68 admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:43:47.360321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be8558-6458-4f7a-9265-f035a6f77c68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:43:47.362156 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: WARNING py.warnings [None req-a7be8558-6458-4f7a-9265-f035a6f77c68 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 14:43:47.362156 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: warnings.warn(msg) Jul 20 14:43:47.362156 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: Jul 20 14:43:47.363384 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be8558-6458-4f7a-9265-f035a6f77c68 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:43:47.443266 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 749/1497] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:43:47 2022] POST /identity/v3/projects => generated 377 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:43:47.450965 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0c466-13bd-4c5c-a1f1-cc4c6585609a admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:43:47.452601 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-aff0c466-13bd-4c5c-a1f1-cc4c6585609a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:43:47.497304 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0c466-13bd-4c5c-a1f1-cc4c6585609a admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:43:47.606474 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd05eed0-16bb-4c2e-93f4-289ae9b4165a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:43:47.607294 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd05eed0-16bb-4c2e-93f4-289ae9b4165a admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:43:47.607619 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd05eed0-16bb-4c2e-93f4-289ae9b4165a admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:43:47.607941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd05eed0-16bb-4c2e-93f4-289ae9b4165a admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:43:47.608710 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd05eed0-16bb-4c2e-93f4-289ae9b4165a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:43:47.609736 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: WARNING py.warnings [None req-cd05eed0-16bb-4c2e-93f4-289ae9b4165a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 14:43:47.609736 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: warnings.warn(msg) Jul 20 14:43:47.609736 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: Jul 20 14:43:47.610959 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd05eed0-16bb-4c2e-93f4-289ae9b4165a admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:43:47.658546 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 749/1498] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:43:47 2022] POST /identity/v3/users => generated 289 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:43:47.663422 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be8558-6458-4f7a-9265-f035a6f77c68 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:43:47.665011 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-a7be8558-6458-4f7a-9265-f035a6f77c68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:43:47.730175 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be8558-6458-4f7a-9265-f035a6f77c68 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:43:47.829996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:43:47.844195 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9508d3-ba87-4356-9ce4-a964366d1154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:43:47.845526 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9508d3-ba87-4356-9ce4-a964366d1154 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:43:47.845526 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9508d3-ba87-4356-9ce4-a964366d1154 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:43:47.845526 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9508d3-ba87-4356-9ce4-a964366d1154 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:43:47.846273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9508d3-ba87-4356-9ce4-a964366d1154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:43:47.847424 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: WARNING py.warnings [None req-9e9508d3-ba87-4356-9ce4-a964366d1154 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 14:43:47.847424 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: warnings.warn(msg) Jul 20 14:43:47.847424 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: Jul 20 14:43:47.848326 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9508d3-ba87-4356-9ce4-a964366d1154 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:43:47.862022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 750/1499] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:43:47 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:43:47.866567 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd05eed0-16bb-4c2e-93f4-289ae9b4165a admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:43:47.868489 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-cd05eed0-16bb-4c2e-93f4-289ae9b4165a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:43:47.883075 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd05eed0-16bb-4c2e-93f4-289ae9b4165a admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:43:47.916877 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:43:48.006823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762076a0-649b-4333-a3cf-65419724e871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:43:48.007544 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762076a0-649b-4333-a3cf-65419724e871 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:43:48.007865 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762076a0-649b-4333-a3cf-65419724e871 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:43:48.008252 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762076a0-649b-4333-a3cf-65419724e871 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:43:48.009141 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762076a0-649b-4333-a3cf-65419724e871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:43:48.010302 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: WARNING py.warnings [None req-762076a0-649b-4333-a3cf-65419724e871 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 14:43:48.010302 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: warnings.warn(msg) Jul 20 14:43:48.010302 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: Jul 20 14:43:48.011338 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762076a0-649b-4333-a3cf-65419724e871 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:43:48.027739 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 750/1500] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:43:47 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:43:48.033144 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9508d3-ba87-4356-9ce4-a964366d1154 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:43:48.034707 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-9e9508d3-ba87-4356-9ce4-a964366d1154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:43:48.085424 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9508d3-ba87-4356-9ce4-a964366d1154 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:43:48.199291 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c416756f-7c07-4091-8a88-c5bb63c28d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:43:48.200189 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416756f-7c07-4091-8a88-c5bb63c28d1b admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:43:48.200463 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416756f-7c07-4091-8a88-c5bb63c28d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:43:48.200721 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416756f-7c07-4091-8a88-c5bb63c28d1b admin admin] PATH_INFO: `/v3/projects/9c0b5f602a83434188ce41e0838b1871/users/6bf5cf27d24346a58d121bcbc56f3b0b/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:43:48.222852 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c416756f-7c07-4091-8a88-c5bb63c28d1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c0b5f602a83434188ce41e0838b1871, user_id=6bf5cf27d24346a58d121bcbc56f3b0b, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:43:48.223937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: WARNING py.warnings [None req-c416756f-7c07-4091-8a88-c5bb63c28d1b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 14:43:48.223937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: warnings.warn(msg) Jul 20 14:43:48.223937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: Jul 20 14:43:48.225846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c416756f-7c07-4091-8a88-c5bb63c28d1b admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:43:48.254885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 751/1501] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:43:48 2022] PUT /identity/v3/projects/9c0b5f602a83434188ce41e0838b1871/users/6bf5cf27d24346a58d121bcbc56f3b0b/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:43:48.270340 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e70e6a-9afd-4e80-b29b-1b122cfa3741 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:43:48.270624 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e70e6a-9afd-4e80-b29b-1b122cfa3741 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:43:48.270977 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e70e6a-9afd-4e80-b29b-1b122cfa3741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:43:48.340770 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-99e70e6a-9afd-4e80-b29b-1b122cfa3741 None None] MFA Rules not processed for user `6bf5cf27d24346a58d121bcbc56f3b0b`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:43:48.342015 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-99e70e6a-9afd-4e80-b29b-1b122cfa3741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:43:48.428013 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 751/1502] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:43:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:43:48.443638 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da27c2-695d-4684-9278-8c835d832394 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:43:48.443969 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da27c2-695d-4684-9278-8c835d832394 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:43:48.444297 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da27c2-695d-4684-9278-8c835d832394 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:43:48.445912 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 752/1503] 10.222.0.15 () {54 vars in 964 bytes} [Wed Jul 20 14:43:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:43:48.451337 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36dc57-7b3a-470f-b6c0-22a30f671976 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:43:48.451680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36dc57-7b3a-470f-b6c0-22a30f671976 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:43:48.451978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36dc57-7b3a-470f-b6c0-22a30f671976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:43:48.485103 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:43:48.544819 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-ca36dc57-7b3a-470f-b6c0-22a30f671976 None None] MFA Rules not processed for user `9618ec9bbe76431e84640b0e5ca90eb1`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:43:48.546211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-ca36dc57-7b3a-470f-b6c0-22a30f671976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:43:48.627669 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 752/1504] 10.222.0.15 () {58 vars in 1061 bytes} [Wed Jul 20 14:43:48 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:43:48.641885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73da27c2-695d-4684-9278-8c835d832394 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:43:48.645082 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-73da27c2-695d-4684-9278-8c835d832394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:43:48.707573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:43:48.772820 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73da27c2-695d-4684-9278-8c835d832394 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:43:48.869013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d411dd5a-369e-47f4-b4ac-98d5b75c4f44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:43:48.869740 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411dd5a-369e-47f4-b4ac-98d5b75c4f44 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:43:48.870180 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411dd5a-369e-47f4-b4ac-98d5b75c4f44 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:43:48.870522 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411dd5a-369e-47f4-b4ac-98d5b75c4f44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:43:48.872822 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-d411dd5a-369e-47f4-b4ac-98d5b75c4f44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:43:48.994359 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d411dd5a-369e-47f4-b4ac-98d5b75c4f44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:43:48.995904 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d411dd5a-369e-47f4-b4ac-98d5b75c4f44 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:43:48.999793 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:43:49.017179 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 753/1505] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:43:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:43:49.070421 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a2897-aed2-4c27-904c-68ac901431ed None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:43:49.070718 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a2897-aed2-4c27-904c-68ac901431ed None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:43:49.071027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a2897-aed2-4c27-904c-68ac901431ed None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:43:49.072761 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 753/1506] 10.222.0.15 () {52 vars in 829 bytes} [Wed Jul 20 14:43:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 14:43:49.077541 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d411dd5a-369e-47f4-b4ac-98d5b75c4f44 service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:43:49.079703 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-d411dd5a-369e-47f4-b4ac-98d5b75c4f44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:43:49.227103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d411dd5a-369e-47f4-b4ac-98d5b75c4f44 service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:43:49.358849 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b8ea8-88b0-4ed1-a3f7-826db5827f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:43:49.359784 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b8ea8-88b0-4ed1-a3f7-826db5827f23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:43:49.360065 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b8ea8-88b0-4ed1-a3f7-826db5827f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:43:49.360343 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b8ea8-88b0-4ed1-a3f7-826db5827f23 admin admin] PATH_INFO: `/v3/users/6bf5cf27d24346a58d121bcbc56f3b0b` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:43:49.386336 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1b8ea8-88b0-4ed1-a3f7-826db5827f23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bf5cf27d24346a58d121bcbc56f3b0b)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:43:49.387669 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: WARNING py.warnings [None req-fe1b8ea8-88b0-4ed1-a3f7-826db5827f23 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 14:43:49.387669 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: warnings.warn(msg) Jul 20 14:43:49.387669 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: Jul 20 14:43:49.389709 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1b8ea8-88b0-4ed1-a3f7-826db5827f23 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:43:49.404313 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:43:49.473376 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fe1b8ea8-88b0-4ed1-a3f7-826db5827f23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bf5cf27d24346a58d121bcbc56f3b0b'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:43:49.495776 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fe1b8ea8-88b0-4ed1-a3f7-826db5827f23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bf5cf27d24346a58d121bcbc56f3b0b'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:43:49.496399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fe1b8ea8-88b0-4ed1-a3f7-826db5827f23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bf5cf27d24346a58d121bcbc56f3b0b'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:43:49.496910 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fe1b8ea8-88b0-4ed1-a3f7-826db5827f23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bf5cf27d24346a58d121bcbc56f3b0b'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:43:49.512121 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fe1b8ea8-88b0-4ed1-a3f7-826db5827f23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bf5cf27d24346a58d121bcbc56f3b0b'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:43:49.532654 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 754/1507] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:43:49 2022] DELETE /identity/v3/users/6bf5cf27d24346a58d121bcbc56f3b0b => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:43:49.538062 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537138c-eaf4-4118-b477-5f468c85b015 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:43:49.538349 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537138c-eaf4-4118-b477-5f468c85b015 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:43:49.538629 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537138c-eaf4-4118-b477-5f468c85b015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:43:49.595040 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-c537138c-eaf4-4118-b477-5f468c85b015 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:43:49.596268 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-c537138c-eaf4-4118-b477-5f468c85b015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:43:49.687614 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:43:49.708793 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 754/1508] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:43:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:43:49.720587 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b8ea8-88b0-4ed1-a3f7-826db5827f23 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:43:49.722273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-fe1b8ea8-88b0-4ed1-a3f7-826db5827f23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:43:49.847321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b8ea8-88b0-4ed1-a3f7-826db5827f23 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:43:49.923723 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:43:49.946532 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679df05-b61e-45c9-90cf-04c1a298a2a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:43:49.947221 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679df05-b61e-45c9-90cf-04c1a298a2a1 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:43:49.947542 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679df05-b61e-45c9-90cf-04c1a298a2a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:43:49.947770 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679df05-b61e-45c9-90cf-04c1a298a2a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:43:49.949427 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-2679df05-b61e-45c9-90cf-04c1a298a2a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:43:50.076880 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2679df05-b61e-45c9-90cf-04c1a298a2a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:43:50.079144 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2679df05-b61e-45c9-90cf-04c1a298a2a1 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:43:50.101066 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 755/1509] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:43:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:43:50.502030 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537138c-eaf4-4118-b477-5f468c85b015 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:43:50.503301 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-c537138c-eaf4-4118-b477-5f468c85b015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:43:50.515587 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537138c-eaf4-4118-b477-5f468c85b015 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:43:50.629493 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a1335-426e-4806-ab75-161c89b5659b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:43:50.630376 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a1335-426e-4806-ab75-161c89b5659b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:43:50.630699 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a1335-426e-4806-ab75-161c89b5659b admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:43:50.631030 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a1335-426e-4806-ab75-161c89b5659b admin admin] PATH_INFO: `/v3/projects/9c0b5f602a83434188ce41e0838b1871` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:43:50.643456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226a1335-426e-4806-ab75-161c89b5659b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c0b5f602a83434188ce41e0838b1871)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:43:50.644634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: WARNING py.warnings [None req-226a1335-426e-4806-ab75-161c89b5659b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 14:43:50.644634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: warnings.warn(msg) Jul 20 14:43:50.644634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: Jul 20 14:43:50.646071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226a1335-426e-4806-ab75-161c89b5659b admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:43:50.722588 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-226a1335-426e-4806-ab75-161c89b5659b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c0b5f602a83434188ce41e0838b1871', 'request_id': 'req-226a1335-426e-4806-ab75-161c89b5659b'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:43:50.729966 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-226a1335-426e-4806-ab75-161c89b5659b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c0b5f602a83434188ce41e0838b1871', 'request_id': 'req-226a1335-426e-4806-ab75-161c89b5659b'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:43:50.746220 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-226a1335-426e-4806-ab75-161c89b5659b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c0b5f602a83434188ce41e0838b1871', 'request_id': 'req-226a1335-426e-4806-ab75-161c89b5659b'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:43:50.777470 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-226a1335-426e-4806-ab75-161c89b5659b admin admin] The token cache is being invalidate because project 9c0b5f602a83434188ce41e0838b1871 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:43:50.777799 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-226a1335-426e-4806-ab75-161c89b5659b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:43:50.778540 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-226a1335-426e-4806-ab75-161c89b5659b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:43:50.780038 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 755/1510] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:43:50 2022] DELETE /identity/v3/projects/9c0b5f602a83434188ce41e0838b1871 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:44:00.098002 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=62 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:44:00.775026 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:45:40.639088 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b5e1c-3d8b-4ab0-8db2-62300605db1b None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:45:40.639755 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b5e1c-3d8b-4ab0-8db2-62300605db1b None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:45:40.639755 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b5e1c-3d8b-4ab0-8db2-62300605db1b None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:45:40.641145 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 14:45:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:45:40.646191 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e2efd-89f6-4901-a9d5-ef499e7e173f None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:45:40.646659 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e2efd-89f6-4901-a9d5-ef499e7e173f None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:45:40.646762 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e2efd-89f6-4901-a9d5-ef499e7e173f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:45:40.868719 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-eb6e2efd-89f6-4901-a9d5-ef499e7e173f None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:45:40.870464 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-eb6e2efd-89f6-4901-a9d5-ef499e7e173f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:45:41.001753 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 14:45:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:45:41.156928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b5e1c-3d8b-4ab0-8db2-62300605db1b None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:45:41.159693 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-937b5e1c-3d8b-4ab0-8db2-62300605db1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:45:41.303770 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b5e1c-3d8b-4ab0-8db2-62300605db1b None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:45:41.309885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a40a30-b4f1-4eb2-b58e-b032a493cbbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:45:41.310594 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a40a30-b4f1-4eb2-b58e-b032a493cbbf service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:45:41.310883 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a40a30-b4f1-4eb2-b58e-b032a493cbbf service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:45:41.311192 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a40a30-b4f1-4eb2-b58e-b032a493cbbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:45:41.321006 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a40a30-b4f1-4eb2-b58e-b032a493cbbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:45:41.322714 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a40a30-b4f1-4eb2-b58e-b032a493cbbf service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:45:41.335138 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 757/1513] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:45:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:45:42.178842 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbecacb8-c46b-492f-ae63-fccdf4d3f786 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:45:42.179466 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbecacb8-c46b-492f-ae63-fccdf4d3f786 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:45:42.179466 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbecacb8-c46b-492f-ae63-fccdf4d3f786 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:45:42.180788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 757/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 14:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:45:42.187598 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4aa470-f7ea-4df6-ba8d-4f77c6f22eba None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:45:42.189114 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4aa470-f7ea-4df6-ba8d-4f77c6f22eba None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:45:42.189114 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4aa470-f7ea-4df6-ba8d-4f77c6f22eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:45:42.210260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-7e4aa470-f7ea-4df6-ba8d-4f77c6f22eba None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:45:42.211682 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-7e4aa470-f7ea-4df6-ba8d-4f77c6f22eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:45:42.223610 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 758/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 14:45:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:45:42.405864 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbecacb8-c46b-492f-ae63-fccdf4d3f786 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:45:42.426425 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbecacb8-c46b-492f-ae63-fccdf4d3f786 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:45:42.432885 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb5fe8-8e79-46d3-af2b-3f7d720f3ad3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:45:42.433530 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb5fe8-8e79-46d3-af2b-3f7d720f3ad3 service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:45:42.433864 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb5fe8-8e79-46d3-af2b-3f7d720f3ad3 service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:45:42.434142 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb5fe8-8e79-46d3-af2b-3f7d720f3ad3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:45:42.444558 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb5fe8-8e79-46d3-af2b-3f7d720f3ad3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:45:42.446260 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb5fe8-8e79-46d3-af2b-3f7d720f3ad3 service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:45:42.453347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315d0af-269c-4c5e-a0b1-2e49b0806a46 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:45:42.453713 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315d0af-269c-4c5e-a0b1-2e49b0806a46 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:45:42.454008 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315d0af-269c-4c5e-a0b1-2e49b0806a46 None None] PATH_INFO: `/v3` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:45:42.455641 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 759/1516] 10.222.0.13 () {54 vars in 925 bytes} [Wed Jul 20 14:45:42 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:45:42.457444 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 758/1517] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:45:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:45:42.461660 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d1375-4fd7-42ae-b110-f5397e23c969 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:45:42.461965 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d1375-4fd7-42ae-b110-f5397e23c969 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:45:42.462245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d1375-4fd7-42ae-b110-f5397e23c969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:45:42.546493 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-343d1375-4fd7-42ae-b110-f5397e23c969 None None] MFA Rules not processed for user `cb857f32bbc04d82bfa681bd5659765f`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:45:42.547893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-343d1375-4fd7-42ae-b110-f5397e23c969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:45:42.618254 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:45:42.639689 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 760/1518] 10.222.0.13 () {58 vars in 1015 bytes} [Wed Jul 20 14:45:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:45:51.831568 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde03dde-32be-4138-8573-cdcc339d147a None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:45:51.832150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde03dde-32be-4138-8573-cdcc339d147a None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:45:51.832150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde03dde-32be-4138-8573-cdcc339d147a None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:45:51.833532 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 759/1519] 10.222.0.15 () {54 vars in 961 bytes} [Wed Jul 20 14:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:45:51.838979 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a77e0-8412-4d8e-ba20-9ab880af12e1 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:45:51.839406 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a77e0-8412-4d8e-ba20-9ab880af12e1 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:45:51.839576 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a77e0-8412-4d8e-ba20-9ab880af12e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:45:51.863344 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-c25a77e0-8412-4d8e-ba20-9ab880af12e1 None None] MFA Rules not processed for user `cb857f32bbc04d82bfa681bd5659765f`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:45:51.864641 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-c25a77e0-8412-4d8e-ba20-9ab880af12e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:45:51.873993 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 761/1520] 10.222.0.15 () {58 vars in 1058 bytes} [Wed Jul 20 14:45:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:45:51.881841 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde03dde-32be-4138-8573-cdcc339d147a None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:45:51.896588 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde03dde-32be-4138-8573-cdcc339d147a None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:45:51.902014 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24702033-960d-40ad-848f-471fcf8dea84 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb857f32bbc04d82bfa681bd5659765f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:45:51.902888 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24702033-960d-40ad-848f-471fcf8dea84 service placement] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:45:51.903006 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24702033-960d-40ad-848f-471fcf8dea84 service placement] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:45:51.903284 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24702033-960d-40ad-848f-471fcf8dea84 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:45:51.913583 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24702033-960d-40ad-848f-471fcf8dea84 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:45:51.915223 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24702033-960d-40ad-848f-471fcf8dea84 service placement] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:45:51.926101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 760/1521] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:45:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:45:53.681523 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf632f50-931f-4c8b-b13d-dab243ace67b None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:45:53.682110 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf632f50-931f-4c8b-b13d-dab243ace67b None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:45:53.682110 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf632f50-931f-4c8b-b13d-dab243ace67b None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:45:53.683864 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 14:45:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:45:53.689871 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4fb8d-56c8-4b5c-92a3-cdb536b372ba None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:45:53.690505 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4fb8d-56c8-4b5c-92a3-cdb536b372ba None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:45:53.690601 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4fb8d-56c8-4b5c-92a3-cdb536b372ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:45:53.710819 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-06d4fb8d-56c8-4b5c-92a3-cdb536b372ba None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:45:53.712181 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-06d4fb8d-56c8-4b5c-92a3-cdb536b372ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:45:53.720837 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 14:45:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:45:53.874077 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf632f50-931f-4c8b-b13d-dab243ace67b None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:45:53.890654 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf632f50-931f-4c8b-b13d-dab243ace67b None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:45:53.896656 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42111cdb-7ac7-4b08-8101-9680281641c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:45:53.897317 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42111cdb-7ac7-4b08-8101-9680281641c4 service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:45:53.897606 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42111cdb-7ac7-4b08-8101-9680281641c4 service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:45:53.897886 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42111cdb-7ac7-4b08-8101-9680281641c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:45:53.907387 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42111cdb-7ac7-4b08-8101-9680281641c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:45:53.908989 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42111cdb-7ac7-4b08-8101-9680281641c4 service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:45:53.919619 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 763/1524] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:45:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:45:54.771400 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0906ba-416f-4937-a087-4bdaa51a3a18 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:45:54.772105 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0906ba-416f-4937-a087-4bdaa51a3a18 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:45:54.772105 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0906ba-416f-4937-a087-4bdaa51a3a18 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:45:54.773245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 14:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:45:54.778471 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a84c3-bb20-4f31-b30c-f71421882489 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:45:54.778922 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a84c3-bb20-4f31-b30c-f71421882489 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:45:54.779036 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a84c3-bb20-4f31-b30c-f71421882489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:45:54.800508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-3e3a84c3-bb20-4f31-b30c-f71421882489 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:45:54.801731 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-3e3a84c3-bb20-4f31-b30c-f71421882489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:45:54.810114 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 14:45:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:45:54.965376 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0906ba-416f-4937-a087-4bdaa51a3a18 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:45:54.983147 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0906ba-416f-4937-a087-4bdaa51a3a18 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:45:54.989120 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe0534-595d-4b59-a6c6-834b4d26fee4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:45:54.989767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe0534-595d-4b59-a6c6-834b4d26fee4 service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:45:54.990044 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe0534-595d-4b59-a6c6-834b4d26fee4 service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:45:54.990346 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe0534-595d-4b59-a6c6-834b4d26fee4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:45:55.000132 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe0534-595d-4b59-a6c6-834b4d26fee4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:45:55.001317 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe0534-595d-4b59-a6c6-834b4d26fee4 service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:45:55.015239 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 763/1527] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:45:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:45:59.460946 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a320a00-4c1d-4b3e-a9c8-182eafa72ab7 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:45:59.461618 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a320a00-4c1d-4b3e-a9c8-182eafa72ab7 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:45:59.461618 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a320a00-4c1d-4b3e-a9c8-182eafa72ab7 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:45:59.462892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 14:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:45:59.468875 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac901b26-35e9-4588-a9c8-1b0a826188e2 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:45:59.469433 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac901b26-35e9-4588-a9c8-1b0a826188e2 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:45:59.469433 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac901b26-35e9-4588-a9c8-1b0a826188e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:45:59.489533 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-ac901b26-35e9-4588-a9c8-1b0a826188e2 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:45:59.490917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-ac901b26-35e9-4588-a9c8-1b0a826188e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:45:59.501107 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 14:45:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:45:59.586484 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a320a00-4c1d-4b3e-a9c8-182eafa72ab7 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:45:59.589043 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-6a320a00-4c1d-4b3e-a9c8-182eafa72ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:45:59.705108 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a320a00-4c1d-4b3e-a9c8-182eafa72ab7 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:45:59.709458 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a20ed1-32d5-40de-bac3-368cd2d4f3e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:45:59.710283 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a20ed1-32d5-40de-bac3-368cd2d4f3e5 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:45:59.710566 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a20ed1-32d5-40de-bac3-368cd2d4f3e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:45:59.710922 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a20ed1-32d5-40de-bac3-368cd2d4f3e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:45:59.724330 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a20ed1-32d5-40de-bac3-368cd2d4f3e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:45:59.726032 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a20ed1-32d5-40de-bac3-368cd2d4f3e5 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:45:59.740723 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 766/1530] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:45:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:46:01.834360 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3162153-0eec-4bae-80f9-a4711dcf55d2 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:46:01.835974 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3162153-0eec-4bae-80f9-a4711dcf55d2 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:46:01.835974 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3162153-0eec-4bae-80f9-a4711dcf55d2 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:46:01.836332 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 14:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:46:01.848284 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c7e74-575e-4af8-8f59-f7fe528031fc None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:46:01.848844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c7e74-575e-4af8-8f59-f7fe528031fc None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:46:01.849007 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c7e74-575e-4af8-8f59-f7fe528031fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:46:01.870000 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-867c7e74-575e-4af8-8f59-f7fe528031fc None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:46:01.871276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-867c7e74-575e-4af8-8f59-f7fe528031fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:46:01.879788 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 14:46:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:46:01.921070 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3162153-0eec-4bae-80f9-a4711dcf55d2 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:46:01.923009 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-b3162153-0eec-4bae-80f9-a4711dcf55d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:46:01.930923 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=94 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:46:01.940693 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3162153-0eec-4bae-80f9-a4711dcf55d2 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:46:01.944775 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e5b11-8d68-4e5f-878b-beacad9f9ae1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:46:01.945467 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e5b11-8d68-4e5f-878b-beacad9f9ae1 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:46:01.945793 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e5b11-8d68-4e5f-878b-beacad9f9ae1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:46:01.946083 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e5b11-8d68-4e5f-878b-beacad9f9ae1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:46:01.956821 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44e5b11-8d68-4e5f-878b-beacad9f9ae1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:46:01.958263 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44e5b11-8d68-4e5f-878b-beacad9f9ae1 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:46:01.971866 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 766/1533] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:46:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:46:06.061939 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177a156-d3cc-408d-a78c-b59800c92bc6 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:46:06.062605 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177a156-d3cc-408d-a78c-b59800c92bc6 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:46:06.062742 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177a156-d3cc-408d-a78c-b59800c92bc6 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:46:06.064372 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 14:46:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:46:06.071440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746664a8-acc9-4bbc-b163-9b8015b64f2b None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:46:06.072006 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746664a8-acc9-4bbc-b163-9b8015b64f2b None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:46:06.072006 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746664a8-acc9-4bbc-b163-9b8015b64f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:46:06.093076 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-746664a8-acc9-4bbc-b163-9b8015b64f2b None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:46:06.094294 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-746664a8-acc9-4bbc-b163-9b8015b64f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:46:06.104422 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 14:46:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:46:06.418555 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177a156-d3cc-408d-a78c-b59800c92bc6 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:46:06.420898 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-0177a156-d3cc-408d-a78c-b59800c92bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:46:06.515387 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177a156-d3cc-408d-a78c-b59800c92bc6 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:46:06.519373 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aada8cc-3bd6-4f9e-828a-be7ed53aa77d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:46:06.520031 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aada8cc-3bd6-4f9e-828a-be7ed53aa77d service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:46:06.520315 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aada8cc-3bd6-4f9e-828a-be7ed53aa77d service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:46:06.520706 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aada8cc-3bd6-4f9e-828a-be7ed53aa77d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:46:06.530621 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aada8cc-3bd6-4f9e-828a-be7ed53aa77d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:46:06.532097 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aada8cc-3bd6-4f9e-828a-be7ed53aa77d service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:46:06.542768 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 769/1536] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:46:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:46:08.617138 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07352c-5f9a-4084-a0ab-bbcbb335cb04 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:46:08.617959 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07352c-5f9a-4084-a0ab-bbcbb335cb04 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:46:08.617959 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07352c-5f9a-4084-a0ab-bbcbb335cb04 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:46:08.619089 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 14:46:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:46:08.624756 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d75556-c267-4af8-ba1c-7177474232aa None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:46:08.625219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d75556-c267-4af8-ba1c-7177474232aa None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:46:08.625317 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d75556-c267-4af8-ba1c-7177474232aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:46:08.645235 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-98d75556-c267-4af8-ba1c-7177474232aa None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:46:08.646451 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-98d75556-c267-4af8-ba1c-7177474232aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:46:08.657626 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 14:46:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:46:08.960275 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07352c-5f9a-4084-a0ab-bbcbb335cb04 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:46:08.962117 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-3e07352c-5f9a-4084-a0ab-bbcbb335cb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:46:08.975884 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07352c-5f9a-4084-a0ab-bbcbb335cb04 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:46:08.979724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a163e-b270-4859-9039-69e3dc7eee8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:46:08.980370 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a163e-b270-4859-9039-69e3dc7eee8a service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:46:08.980644 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a163e-b270-4859-9039-69e3dc7eee8a service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:46:08.980922 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a163e-b270-4859-9039-69e3dc7eee8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:46:08.990859 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a163e-b270-4859-9039-69e3dc7eee8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:46:08.992305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a163e-b270-4859-9039-69e3dc7eee8a service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:46:09.002095 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 769/1539] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:46:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:46:18.639552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=92 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:46:18.997694 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=14 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:42.454923 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817a90b-c793-4a7e-b26c-f722bcacd80e None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:42.456275 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817a90b-c793-4a7e-b26c-f722bcacd80e None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:42.456779 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817a90b-c793-4a7e-b26c-f722bcacd80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:42.488572 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-4817a90b-c793-4a7e-b26c-f722bcacd80e None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:42.490224 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-4817a90b-c793-4a7e-b26c-f722bcacd80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:42.509077 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 771/1540] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:42.518890 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a72d1-0f2f-4271-bd56-4751b761f194 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:42.519522 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a72d1-0f2f-4271-bd56-4751b761f194 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:42.519522 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a72d1-0f2f-4271-bd56-4751b761f194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:42.543558 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-d12a72d1-0f2f-4271-bd56-4751b761f194 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:42.545153 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-d12a72d1-0f2f-4271-bd56-4751b761f194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:42.553518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 770/1541] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:42.559389 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817a90b-c793-4a7e-b26c-f722bcacd80e None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:42.573852 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817a90b-c793-4a7e-b26c-f722bcacd80e None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:42.588997 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02284391-d21e-4f5f-9157-6548659b7aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:42.589695 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02284391-d21e-4f5f-9157-6548659b7aba admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:42.589981 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02284391-d21e-4f5f-9157-6548659b7aba admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:42.590320 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02284391-d21e-4f5f-9157-6548659b7aba admin admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:42.591097 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02284391-d21e-4f5f-9157-6548659b7aba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:47:42.591416 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02284391-d21e-4f5f-9157-6548659b7aba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:42.592482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: WARNING py.warnings [None req-02284391-d21e-4f5f-9157-6548659b7aba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 14:47:42.592482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: warnings.warn(msg) Jul 20 14:47:42.592482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: Jul 20 14:47:42.593382 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02284391-d21e-4f5f-9157-6548659b7aba admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:42.612990 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99c051-4a3d-4611-86fc-9e0e47bffad9 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:42.613660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99c051-4a3d-4611-86fc-9e0e47bffad9 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:42.614152 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99c051-4a3d-4611-86fc-9e0e47bffad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:42.617240 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 772/1542] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:47:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:47:42.635075 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad4efc-7672-4659-b214-abda3112ce0c None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:42.635075 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad4efc-7672-4659-b214-abda3112ce0c None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:42.635075 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad4efc-7672-4659-b214-abda3112ce0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:42.646882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-2e99c051-4a3d-4611-86fc-9e0e47bffad9 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:42.651389 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-2e99c051-4a3d-4611-86fc-9e0e47bffad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:42.660597 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 771/1543] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:47:42.661768 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99c051-4a3d-4611-86fc-9e0e47bffad9 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:42.676819 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-f3ad4efc-7672-4659-b214-abda3112ce0c None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:42.678051 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-f3ad4efc-7672-4659-b214-abda3112ce0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:42.680811 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99c051-4a3d-4611-86fc-9e0e47bffad9 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:42.686229 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3ee74-0455-4839-b9ae-19ffafe4eaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:42.687294 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3ee74-0455-4839-b9ae-19ffafe4eaca admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:42.687914 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3ee74-0455-4839-b9ae-19ffafe4eaca admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:42.688417 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3ee74-0455-4839-b9ae-19ffafe4eaca admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:42.689444 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3ee74-0455-4839-b9ae-19ffafe4eaca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:42.690709 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: WARNING py.warnings [None req-aff3ee74-0455-4839-b9ae-19ffafe4eaca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 14:47:42.690709 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: warnings.warn(msg) Jul 20 14:47:42.690709 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: Jul 20 14:47:42.692307 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3ee74-0455-4839-b9ae-19ffafe4eaca admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:42.697831 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 773/1544] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:47:42.699434 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bff592-4b17-4149-9f83-00d0094b00f4 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:42.699434 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bff592-4b17-4149-9f83-00d0094b00f4 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:42.699666 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bff592-4b17-4149-9f83-00d0094b00f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:42.728786 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-c8bff592-4b17-4149-9f83-00d0094b00f4 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:42.730259 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-c8bff592-4b17-4149-9f83-00d0094b00f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:42.737343 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 772/1545] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:47:42 2022] POST /identity/v3/projects => generated 379 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:47:42.739132 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156afcf5-31dd-4628-a5a0-1666dc795fcc None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:42.739662 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156afcf5-31dd-4628-a5a0-1666dc795fcc None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:42.740210 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156afcf5-31dd-4628-a5a0-1666dc795fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:42.779859 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-156afcf5-31dd-4628-a5a0-1666dc795fcc None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:42.781071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-156afcf5-31dd-4628-a5a0-1666dc795fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:42.824033 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 774/1546] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:42.826951 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f8395-5bad-4a85-afea-6592f7653ce2 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:42.827645 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f8395-5bad-4a85-afea-6592f7653ce2 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:42.828401 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f8395-5bad-4a85-afea-6592f7653ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:42.854449 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-940f8395-5bad-4a85-afea-6592f7653ce2 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:42.855721 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-940f8395-5bad-4a85-afea-6592f7653ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:42.860349 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 773/1547] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:42.864472 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 775/1548] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:42.864600 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156afcf5-31dd-4628-a5a0-1666dc795fcc None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:42.865216 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696d43d-eeb5-4aaf-95cf-c23a7e2a7bf6 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:42.865505 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696d43d-eeb5-4aaf-95cf-c23a7e2a7bf6 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:42.865781 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696d43d-eeb5-4aaf-95cf-c23a7e2a7bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:42.877025 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156afcf5-31dd-4628-a5a0-1666dc795fcc None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:42.883255 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0977ed9-1686-4e4c-9f05-6a968558297f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:42.883951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0977ed9-1686-4e4c-9f05-6a968558297f admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:42.884263 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0977ed9-1686-4e4c-9f05-6a968558297f admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:42.884543 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0977ed9-1686-4e4c-9f05-6a968558297f admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:42.885211 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-7696d43d-eeb5-4aaf-95cf-c23a7e2a7bf6 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:42.885345 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0977ed9-1686-4e4c-9f05-6a968558297f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:42.886504 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-7696d43d-eeb5-4aaf-95cf-c23a7e2a7bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:42.887197 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0977ed9-1686-4e4c-9f05-6a968558297f admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:42.894520 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 776/1549] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:42.895757 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed2db0-3ea7-417f-8b28-08197e8063b2 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:42.896042 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed2db0-3ea7-417f-8b28-08197e8063b2 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:42.896316 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed2db0-3ea7-417f-8b28-08197e8063b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:42.913139 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 774/1550] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:47:42 2022] POST /identity/v3/users => generated 290 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:47:42.914559 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811b37e-369b-429c-949b-8eeb02b2ccf2 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:42.914913 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811b37e-369b-429c-949b-8eeb02b2ccf2 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:42.915285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811b37e-369b-429c-949b-8eeb02b2ccf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:42.920064 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-a4ed2db0-3ea7-417f-8b28-08197e8063b2 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:42.921282 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-a4ed2db0-3ea7-417f-8b28-08197e8063b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:42.929366 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 777/1551] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:42.930201 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed2db0-3ea7-417f-8b28-08197e8063b2 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:42.937190 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-b811b37e-369b-429c-949b-8eeb02b2ccf2 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:42.939225 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-b811b37e-369b-429c-949b-8eeb02b2ccf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:42.942186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed2db0-3ea7-417f-8b28-08197e8063b2 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:42.946865 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23562e-9710-4c39-9ead-e4b502ebdb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:42.947569 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23562e-9710-4c39-9ead-e4b502ebdb7b admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:42.947876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 775/1552] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:42.948013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23562e-9710-4c39-9ead-e4b502ebdb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:42.948646 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811b37e-369b-429c-949b-8eeb02b2ccf2 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:42.955300 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23562e-9710-4c39-9ead-e4b502ebdb7b admin admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:42.956125 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23562e-9710-4c39-9ead-e4b502ebdb7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:47:42.956449 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23562e-9710-4c39-9ead-e4b502ebdb7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:42.958103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23562e-9710-4c39-9ead-e4b502ebdb7b admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:42.958792 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811b37e-369b-429c-949b-8eeb02b2ccf2 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:42.963992 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06132bf-4035-4f9f-a589-b164ba0eb1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:42.964673 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06132bf-4035-4f9f-a589-b164ba0eb1e8 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:42.964965 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06132bf-4035-4f9f-a589-b164ba0eb1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:42.965246 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06132bf-4035-4f9f-a589-b164ba0eb1e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:42.965997 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06132bf-4035-4f9f-a589-b164ba0eb1e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:47:42.966305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06132bf-4035-4f9f-a589-b164ba0eb1e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:42.967945 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06132bf-4035-4f9f-a589-b164ba0eb1e8 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:42.970704 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 778/1553] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:47:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:47:42.972138 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12543d98-47e1-40a9-a608-f3c2019a59db None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:42.972538 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12543d98-47e1-40a9-a608-f3c2019a59db None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:42.972823 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12543d98-47e1-40a9-a608-f3c2019a59db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:42.981805 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 776/1554] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:47:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:47:42.983030 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e307d-10ac-4192-b28f-40599e587124 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:42.983515 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e307d-10ac-4192-b28f-40599e587124 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:42.983801 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e307d-10ac-4192-b28f-40599e587124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:42.993803 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-12543d98-47e1-40a9-a608-f3c2019a59db None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:42.994112 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-12543d98-47e1-40a9-a608-f3c2019a59db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:43.004852 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 779/1555] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:43.006626 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd385c9-e066-4ec6-a64d-b9e07a67aa34 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:43.007140 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd385c9-e066-4ec6-a64d-b9e07a67aa34 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:43.007501 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-148e307d-10ac-4192-b28f-40599e587124 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:43.007691 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd385c9-e066-4ec6-a64d-b9e07a67aa34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:43.009116 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-148e307d-10ac-4192-b28f-40599e587124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:43.017537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 777/1556] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:43.019261 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e307d-10ac-4192-b28f-40599e587124 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:43.026688 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-3cd385c9-e066-4ec6-a64d-b9e07a67aa34 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:43.027943 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-3cd385c9-e066-4ec6-a64d-b9e07a67aa34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:43.028404 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e307d-10ac-4192-b28f-40599e587124 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:43.033445 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d96dcf-14e0-46d4-a640-910372caf2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:43.034124 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d96dcf-14e0-46d4-a640-910372caf2d7 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:43.034406 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d96dcf-14e0-46d4-a640-910372caf2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:43.034676 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d96dcf-14e0-46d4-a640-910372caf2d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:43.035463 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d96dcf-14e0-46d4-a640-910372caf2d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:43.036410 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 780/1557] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:43.040924 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d96dcf-14e0-46d4-a640-910372caf2d7 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:43.041122 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e908151-481c-496b-8504-1b67e1db3c91 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:43.041122 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e908151-481c-496b-8504-1b67e1db3c91 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:43.041122 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e908151-481c-496b-8504-1b67e1db3c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:43.049560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 778/1558] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:47:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:47:43.050729 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b915382-afc1-42f0-bcfd-db6113d8d829 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:43.051098 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b915382-afc1-42f0-bcfd-db6113d8d829 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:43.051381 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b915382-afc1-42f0-bcfd-db6113d8d829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:43.060373 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-8e908151-481c-496b-8504-1b67e1db3c91 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:43.061593 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-8e908151-481c-496b-8504-1b67e1db3c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:43.069956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 781/1559] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:43.070249 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-9b915382-afc1-42f0-bcfd-db6113d8d829 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:43.070752 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e908151-481c-496b-8504-1b67e1db3c91 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:43.071488 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-9b915382-afc1-42f0-bcfd-db6113d8d829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:43.080005 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 779/1560] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:43.080301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e908151-481c-496b-8504-1b67e1db3c91 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:43.080805 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b915382-afc1-42f0-bcfd-db6113d8d829 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:43.085710 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517dccf4-f9d1-407d-a80e-d9928f0b9949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:43.086212 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dccf4-f9d1-407d-a80e-d9928f0b9949 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:43.086498 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dccf4-f9d1-407d-a80e-d9928f0b9949 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:43.086917 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dccf4-f9d1-407d-a80e-d9928f0b9949 admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:43.087676 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517dccf4-f9d1-407d-a80e-d9928f0b9949 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:43.089594 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517dccf4-f9d1-407d-a80e-d9928f0b9949 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:43.090630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b915382-afc1-42f0-bcfd-db6113d8d829 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:43.096113 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165b79b-1d46-4345-978d-2f6d8df0bca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:43.096946 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165b79b-1d46-4345-978d-2f6d8df0bca0 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:43.097239 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165b79b-1d46-4345-978d-2f6d8df0bca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:43.097523 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165b79b-1d46-4345-978d-2f6d8df0bca0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:43.098235 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6165b79b-1d46-4345-978d-2f6d8df0bca0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:43.100198 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6165b79b-1d46-4345-978d-2f6d8df0bca0 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:43.122347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 782/1561] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:47:43 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:47:43.123626 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517dccf4-f9d1-407d-a80e-d9928f0b9949 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:43.134788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 780/1562] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:47:43 2022] POST /identity/v3/projects => generated 393 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:47:43.134788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117637d-f6cd-45af-a8a4-9b923bf0630f None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:43.134788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117637d-f6cd-45af-a8a4-9b923bf0630f None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:43.134788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117637d-f6cd-45af-a8a4-9b923bf0630f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:43.152992 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-9117637d-f6cd-45af-a8a4-9b923bf0630f None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:43.154212 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-9117637d-f6cd-45af-a8a4-9b923bf0630f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:43.189367 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:43.201724 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517dccf4-f9d1-407d-a80e-d9928f0b9949 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:43.205479 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:43.223175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 781/1563] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:43.223175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117637d-f6cd-45af-a8a4-9b923bf0630f None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:43.233668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117637d-f6cd-45af-a8a4-9b923bf0630f None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:43.239712 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74355959-5cdf-44c8-b0be-be42d2f226b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:43.240616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74355959-5cdf-44c8-b0be-be42d2f226b5 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:43.241053 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74355959-5cdf-44c8-b0be-be42d2f226b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:43.241530 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74355959-5cdf-44c8-b0be-be42d2f226b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:43.242490 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74355959-5cdf-44c8-b0be-be42d2f226b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:47:43.242953 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74355959-5cdf-44c8-b0be-be42d2f226b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:43.244959 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74355959-5cdf-44c8-b0be-be42d2f226b5 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:43.258129 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 782/1564] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:47:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:47:43.259026 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74355959-5cdf-44c8-b0be-be42d2f226b5 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:43.266797 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06875fd1-f812-402a-959e-ad66be244f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:43.267548 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06875fd1-f812-402a-959e-ad66be244f5d admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:43.267833 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06875fd1-f812-402a-959e-ad66be244f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:43.268106 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06875fd1-f812-402a-959e-ad66be244f5d admin admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:43.268876 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06875fd1-f812-402a-959e-ad66be244f5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:47:43.269207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06875fd1-f812-402a-959e-ad66be244f5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:43.270882 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06875fd1-f812-402a-959e-ad66be244f5d admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:43.273334 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74355959-5cdf-44c8-b0be-be42d2f226b5 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:43.277966 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c337d8b-0a86-4d49-8412-b99713697d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:43.278758 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c337d8b-0a86-4d49-8412-b99713697d07 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:43.278949 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c337d8b-0a86-4d49-8412-b99713697d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:43.279228 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c337d8b-0a86-4d49-8412-b99713697d07 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:43.280216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c337d8b-0a86-4d49-8412-b99713697d07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:43.282026 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c337d8b-0a86-4d49-8412-b99713697d07 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:43.285099 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 783/1565] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:47:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:47:43.287009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06875fd1-f812-402a-959e-ad66be244f5d admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:43.294093 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 783/1566] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:47:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:47:43.294941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c337d8b-0a86-4d49-8412-b99713697d07 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:43.302862 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06875fd1-f812-402a-959e-ad66be244f5d admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:43.304906 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c337d8b-0a86-4d49-8412-b99713697d07 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:43.307740 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689bcea6-0754-420a-85b5-545307067424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:43.308449 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689bcea6-0754-420a-85b5-545307067424 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:43.308736 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689bcea6-0754-420a-85b5-545307067424 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:43.309023 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689bcea6-0754-420a-85b5-545307067424 admin admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:43.309448 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd293f0-2a0b-4e10-b98a-a4e5658b1d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:43.309787 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689bcea6-0754-420a-85b5-545307067424 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:47:43.310100 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689bcea6-0754-420a-85b5-545307067424 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:43.310219 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd293f0-2a0b-4e10-b98a-a4e5658b1d57 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:43.310376 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd293f0-2a0b-4e10-b98a-a4e5658b1d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:43.310660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd293f0-2a0b-4e10-b98a-a4e5658b1d57 admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:43.311414 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd293f0-2a0b-4e10-b98a-a4e5658b1d57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:47:43.311723 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd293f0-2a0b-4e10-b98a-a4e5658b1d57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:43.311835 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689bcea6-0754-420a-85b5-545307067424 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:43.313368 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd293f0-2a0b-4e10-b98a-a4e5658b1d57 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:43.325153 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 784/1567] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:47:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:47:43.325444 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 784/1568] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:47:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:47:43.325990 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd293f0-2a0b-4e10-b98a-a4e5658b1d57 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:43.326196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689bcea6-0754-420a-85b5-545307067424 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:43.342416 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd293f0-2a0b-4e10-b98a-a4e5658b1d57 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:43.344452 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689bcea6-0754-420a-85b5-545307067424 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:43.347972 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50122c33-a8e8-49d2-a157-70d8ad6d3f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:43.348699 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50122c33-a8e8-49d2-a157-70d8ad6d3f02 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:43.348987 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50122c33-a8e8-49d2-a157-70d8ad6d3f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:43.349133 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2c433-acaa-4e1a-b221-852692266a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:43.349262 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50122c33-a8e8-49d2-a157-70d8ad6d3f02 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:43.349816 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2c433-acaa-4e1a-b221-852692266a29 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:43.349939 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50122c33-a8e8-49d2-a157-70d8ad6d3f02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:43.350104 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2c433-acaa-4e1a-b221-852692266a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:43.350385 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2c433-acaa-4e1a-b221-852692266a29 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:43.351111 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c2c433-acaa-4e1a-b221-852692266a29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:43.351946 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50122c33-a8e8-49d2-a157-70d8ad6d3f02 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:43.352137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: WARNING py.warnings [None req-b5c2c433-acaa-4e1a-b221-852692266a29 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 14:47:43.352137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: warnings.warn(msg) Jul 20 14:47:43.352137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: Jul 20 14:47:43.353372 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c2c433-acaa-4e1a-b221-852692266a29 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:43.375155 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 785/1569] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:47:43 2022] POST /identity/v3/users => generated 293 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:47:43.376105 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50122c33-a8e8-49d2-a157-70d8ad6d3f02 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:43.376855 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 785/1570] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:47:43 2022] POST /identity/v3/users => generated 297 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:47:43.379950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2c433-acaa-4e1a-b221-852692266a29 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:43.386485 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50122c33-a8e8-49d2-a157-70d8ad6d3f02 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:43.390220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2c433-acaa-4e1a-b221-852692266a29 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:43.391325 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfce959-adf7-40b9-85af-01e363bb29dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:43.392044 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfce959-adf7-40b9-85af-01e363bb29dc admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:43.392300 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfce959-adf7-40b9-85af-01e363bb29dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:43.392578 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfce959-adf7-40b9-85af-01e363bb29dc admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:43.393321 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfce959-adf7-40b9-85af-01e363bb29dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:47:43.393639 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfce959-adf7-40b9-85af-01e363bb29dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:43.395292 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfce959-adf7-40b9-85af-01e363bb29dc admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:43.396401 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545339f-32fd-446a-a0cd-36ac352e59f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:43.397092 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545339f-32fd-446a-a0cd-36ac352e59f8 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:43.397371 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545339f-32fd-446a-a0cd-36ac352e59f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:43.397770 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545339f-32fd-446a-a0cd-36ac352e59f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:43.398477 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e545339f-32fd-446a-a0cd-36ac352e59f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:43.400607 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e545339f-32fd-446a-a0cd-36ac352e59f8 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:43.409791 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 786/1571] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:47:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:47:43.410913 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfce959-adf7-40b9-85af-01e363bb29dc admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:43.424394 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfce959-adf7-40b9-85af-01e363bb29dc admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:43.429800 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a1aab1-fd06-4557-b14e-2614e3611ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:43.430491 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1aab1-fd06-4557-b14e-2614e3611ea2 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:43.430779 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1aab1-fd06-4557-b14e-2614e3611ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:43.431363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1aab1-fd06-4557-b14e-2614e3611ea2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:43.432086 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a1aab1-fd06-4557-b14e-2614e3611ea2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:43.434002 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a1aab1-fd06-4557-b14e-2614e3611ea2 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:43.435961 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 786/1572] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:47:43 2022] POST /identity/v3/projects => generated 387 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:47:43.439973 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545339f-32fd-446a-a0cd-36ac352e59f8 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:43.464281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 787/1573] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:47:43 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:47:43.465588 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a1aab1-fd06-4557-b14e-2614e3611ea2 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:43.505938 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545339f-32fd-446a-a0cd-36ac352e59f8 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:43.534862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a1aab1-fd06-4557-b14e-2614e3611ea2 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:43.539721 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc8a0bb-fceb-4362-b49a-102f663a57d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:43.540385 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8a0bb-fceb-4362-b49a-102f663a57d1 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:43.540677 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8a0bb-fceb-4362-b49a-102f663a57d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:43.540954 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8a0bb-fceb-4362-b49a-102f663a57d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:43.541657 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc8a0bb-fceb-4362-b49a-102f663a57d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:43.543631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc8a0bb-fceb-4362-b49a-102f663a57d1 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:43.566491 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:43.575040 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207943af-f11b-4f50-82ad-31f9e967c635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:43.576647 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207943af-f11b-4f50-82ad-31f9e967c635 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:43.577932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207943af-f11b-4f50-82ad-31f9e967c635 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:43.578582 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207943af-f11b-4f50-82ad-31f9e967c635 admin admin] PATH_INFO: `/v3/projects/3d28cd25ee0c476e8ab3b24d7f2f9ab9/users/5373c0299ea64c9f872e3bc04bfb0b74/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:43.587961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 788/1574] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:47:43 2022] POST /identity/v3/projects => generated 391 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:47:43.587961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc8a0bb-fceb-4362-b49a-102f663a57d1 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:43.598366 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207943af-f11b-4f50-82ad-31f9e967c635 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d28cd25ee0c476e8ab3b24d7f2f9ab9, user_id=5373c0299ea64c9f872e3bc04bfb0b74, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:43.600740 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207943af-f11b-4f50-82ad-31f9e967c635 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:43.615705 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 787/1575] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:47:43 2022] PUT /identity/v3/projects/3d28cd25ee0c476e8ab3b24d7f2f9ab9/users/5373c0299ea64c9f872e3bc04bfb0b74/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:47:43.616579 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207943af-f11b-4f50-82ad-31f9e967c635 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:43.625654 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:43.666682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc8a0bb-fceb-4362-b49a-102f663a57d1 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:43.697384 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207943af-f11b-4f50-82ad-31f9e967c635 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:43.705535 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb169fe-0835-46ec-a874-d2942e31a41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:43.706498 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb169fe-0835-46ec-a874-d2942e31a41e admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:43.706779 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb169fe-0835-46ec-a874-d2942e31a41e admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:43.707076 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb169fe-0835-46ec-a874-d2942e31a41e admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:43.707823 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb169fe-0835-46ec-a874-d2942e31a41e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:43.709623 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb169fe-0835-46ec-a874-d2942e31a41e admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:43.720562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 788/1576] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:47:43 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:47:43.724272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb169fe-0835-46ec-a874-d2942e31a41e admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:43.734976 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16002cf3-fa23-4810-9d82-52bba66dc235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:43.735252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb169fe-0835-46ec-a874-d2942e31a41e admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:43.735624 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16002cf3-fa23-4810-9d82-52bba66dc235 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:43.736009 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16002cf3-fa23-4810-9d82-52bba66dc235 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:43.736289 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16002cf3-fa23-4810-9d82-52bba66dc235 admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:43.737207 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16002cf3-fa23-4810-9d82-52bba66dc235 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:43.739311 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16002cf3-fa23-4810-9d82-52bba66dc235 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:43.740067 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcffd47-ea1a-4089-92c4-015256026012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:43.740787 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcffd47-ea1a-4089-92c4-015256026012 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:43.741072 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcffd47-ea1a-4089-92c4-015256026012 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:43.741347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcffd47-ea1a-4089-92c4-015256026012 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:43.742081 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcffd47-ea1a-4089-92c4-015256026012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:43.743826 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcffd47-ea1a-4089-92c4-015256026012 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:43.754932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 789/1577] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:47:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:47:43.755865 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcffd47-ea1a-4089-92c4-015256026012 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:43.766190 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcffd47-ea1a-4089-92c4-015256026012 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:43.773242 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 789/1578] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:47:43 2022] POST /identity/v3/projects => generated 367 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:47:43.773394 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b441b1-87ce-4b8d-bd59-f84f08345f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:43.773509 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16002cf3-fa23-4810-9d82-52bba66dc235 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:43.773623 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b441b1-87ce-4b8d-bd59-f84f08345f6d admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:43.773623 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b441b1-87ce-4b8d-bd59-f84f08345f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:43.773623 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b441b1-87ce-4b8d-bd59-f84f08345f6d admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:43.773623 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b441b1-87ce-4b8d-bd59-f84f08345f6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:43.774808 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b441b1-87ce-4b8d-bd59-f84f08345f6d admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:43.802517 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 790/1579] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:47:43 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:47:43.803360 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b441b1-87ce-4b8d-bd59-f84f08345f6d admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:43.842736 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16002cf3-fa23-4810-9d82-52bba66dc235 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:43.879009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b441b1-87ce-4b8d-bd59-f84f08345f6d admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:43.885532 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5eea85-f0b0-429c-847e-da82585e6a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:43.886401 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5eea85-f0b0-429c-847e-da82585e6a66 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:43.886713 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5eea85-f0b0-429c-847e-da82585e6a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:43.887182 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5eea85-f0b0-429c-847e-da82585e6a66 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:43.887896 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5eea85-f0b0-429c-847e-da82585e6a66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:43.889860 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5eea85-f0b0-429c-847e-da82585e6a66 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:43.900132 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338dd87f-27d1-4a03-a207-8b4fac375194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:43.900815 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338dd87f-27d1-4a03-a207-8b4fac375194 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:43.901075 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338dd87f-27d1-4a03-a207-8b4fac375194 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:43.901348 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338dd87f-27d1-4a03-a207-8b4fac375194 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:43.902109 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338dd87f-27d1-4a03-a207-8b4fac375194 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:43.904366 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338dd87f-27d1-4a03-a207-8b4fac375194 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:43.911553 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 791/1580] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:47:43 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:47:43.912461 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5eea85-f0b0-429c-847e-da82585e6a66 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:43.920964 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:43.923624 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5eea85-f0b0-429c-847e-da82585e6a66 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:43.928231 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 790/1581] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:47:43 2022] POST /identity/v3/users => generated 294 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:47:43.930336 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07f2b9-2914-49af-861c-ae7258d5b409 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:43.930908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07f2b9-2914-49af-861c-ae7258d5b409 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:43.931569 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07f2b9-2914-49af-861c-ae7258d5b409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:43.931713 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2825f1-da3e-40e1-8644-ca004608bc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:43.932323 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2825f1-da3e-40e1-8644-ca004608bc87 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:43.932610 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2825f1-da3e-40e1-8644-ca004608bc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:43.932886 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2825f1-da3e-40e1-8644-ca004608bc87 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:43.933578 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2825f1-da3e-40e1-8644-ca004608bc87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:43.935530 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2825f1-da3e-40e1-8644-ca004608bc87 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:43.948861 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-1b07f2b9-2914-49af-861c-ae7258d5b409 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:43.950337 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-1b07f2b9-2914-49af-861c-ae7258d5b409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:43.955908 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 792/1582] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:47:43 2022] POST /identity/v3/users => generated 296 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:47:43.956939 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2825f1-da3e-40e1-8644-ca004608bc87 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:43.958502 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 791/1583] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:43.959211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b07f2b9-2914-49af-861c-ae7258d5b409 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:43.971351 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2825f1-da3e-40e1-8644-ca004608bc87 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:43.973134 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b07f2b9-2914-49af-861c-ae7258d5b409 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:43.975918 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0211b725-7c65-4e94-a495-f8794eef887f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:43.976607 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211b725-7c65-4e94-a495-f8794eef887f admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:43.976888 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211b725-7c65-4e94-a495-f8794eef887f admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:43.977165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211b725-7c65-4e94-a495-f8794eef887f admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:43.977890 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0211b725-7c65-4e94-a495-f8794eef887f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:43.978315 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28e549-3955-4c2c-917a-02a3961a8c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:43.978986 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28e549-3955-4c2c-917a-02a3961a8c7f admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:43.979290 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28e549-3955-4c2c-917a-02a3961a8c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:43.979517 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0211b725-7c65-4e94-a495-f8794eef887f admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:43.979614 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28e549-3955-4c2c-917a-02a3961a8c7f admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:43.980422 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28e549-3955-4c2c-917a-02a3961a8c7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:43.982169 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28e549-3955-4c2c-917a-02a3961a8c7f admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:43.990386 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 793/1584] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:47:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:47:43.991223 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0211b725-7c65-4e94-a495-f8794eef887f admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:43.992743 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 792/1585] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:47:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:47:43.993716 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28e549-3955-4c2c-917a-02a3961a8c7f admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:43.996301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:44.001897 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0211b725-7c65-4e94-a495-f8794eef887f admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:44.007501 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28e549-3955-4c2c-917a-02a3961a8c7f admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:44.008353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4edaba-0972-4896-8395-27dfa1c27ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:44.009190 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4edaba-0972-4896-8395-27dfa1c27ca4 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:44.009539 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4edaba-0972-4896-8395-27dfa1c27ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:44.009887 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4edaba-0972-4896-8395-27dfa1c27ca4 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:44.010645 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4edaba-0972-4896-8395-27dfa1c27ca4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:44.012285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17b746-9ad6-471e-9687-820a4e10b760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:44.012712 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4edaba-0972-4896-8395-27dfa1c27ca4 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:44.012952 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17b746-9ad6-471e-9687-820a4e10b760 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:44.013238 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17b746-9ad6-471e-9687-820a4e10b760 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:44.013537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17b746-9ad6-471e-9687-820a4e10b760 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:44.014205 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f17b746-9ad6-471e-9687-820a4e10b760 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:44.016264 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f17b746-9ad6-471e-9687-820a4e10b760 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:44.033662 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 794/1586] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:47:43 2022] POST /identity/v3/users => generated 284 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:47:44.034466 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4edaba-0972-4896-8395-27dfa1c27ca4 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:44.040217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 793/1587] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:47:43 2022] POST /identity/v3/users => generated 286 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:47:44.040217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17b746-9ad6-471e-9687-820a4e10b760 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:44.044190 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4edaba-0972-4896-8395-27dfa1c27ca4 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:44.049026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe64a381-82f7-4d98-b17b-a7ec9c188e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:44.049170 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17b746-9ad6-471e-9687-820a4e10b760 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:44.049829 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64a381-82f7-4d98-b17b-a7ec9c188e49 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:44.050181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64a381-82f7-4d98-b17b-a7ec9c188e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:44.050522 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64a381-82f7-4d98-b17b-a7ec9c188e49 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:44.051475 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe64a381-82f7-4d98-b17b-a7ec9c188e49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:44.053143 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe64a381-82f7-4d98-b17b-a7ec9c188e49 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:44.054238 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f8f47-2185-42b9-b84b-13116be21370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:44.054951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f8f47-2185-42b9-b84b-13116be21370 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:44.055305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f8f47-2185-42b9-b84b-13116be21370 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:44.055589 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f8f47-2185-42b9-b84b-13116be21370 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:44.056322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f8f47-2185-42b9-b84b-13116be21370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:44.057977 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f8f47-2185-42b9-b84b-13116be21370 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:44.063697 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 795/1588] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:47:44.067486 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe64a381-82f7-4d98-b17b-a7ec9c188e49 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:44.069075 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 794/1589] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:47:44.069943 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f8f47-2185-42b9-b84b-13116be21370 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:44.077753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe64a381-82f7-4d98-b17b-a7ec9c188e49 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:44.082467 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2f2f4d-7829-4785-aa93-482b1077aa92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:44.083182 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f2f4d-7829-4785-aa93-482b1077aa92 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:44.083491 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f2f4d-7829-4785-aa93-482b1077aa92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:44.083799 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f2f4d-7829-4785-aa93-482b1077aa92 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:44.084517 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2f2f4d-7829-4785-aa93-482b1077aa92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:44.086103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2f2f4d-7829-4785-aa93-482b1077aa92 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:44.097151 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 796/1590] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:47:44.097737 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2f2f4d-7829-4785-aa93-482b1077aa92 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:44.108055 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2f2f4d-7829-4785-aa93-482b1077aa92 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:44.112739 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99c57b-2ac8-4ae8-88bd-2755f946e675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:44.113632 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99c57b-2ac8-4ae8-88bd-2755f946e675 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:44.113910 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99c57b-2ac8-4ae8-88bd-2755f946e675 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:44.114778 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99c57b-2ac8-4ae8-88bd-2755f946e675 admin admin] PATH_INFO: `/v3/projects/5d984345e0d340818c9030f70bace6d3/users/e3574b5b8f3241b192b0b810122d4769/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:44.126908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f8f47-2185-42b9-b84b-13116be21370 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:44.129840 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99c57b-2ac8-4ae8-88bd-2755f946e675 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d984345e0d340818c9030f70bace6d3, user_id=e3574b5b8f3241b192b0b810122d4769, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:44.130622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da61be-c780-4be5-a4b3-f679d189c849 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:44.131314 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da61be-c780-4be5-a4b3-f679d189c849 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:44.131598 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da61be-c780-4be5-a4b3-f679d189c849 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:44.131877 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da61be-c780-4be5-a4b3-f679d189c849 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:44.132929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99c57b-2ac8-4ae8-88bd-2755f946e675 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:44.141516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22da61be-c780-4be5-a4b3-f679d189c849 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:44.143068 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22da61be-c780-4be5-a4b3-f679d189c849 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:44.146314 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 797/1591] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:47:44 2022] PUT /identity/v3/projects/5d984345e0d340818c9030f70bace6d3/users/e3574b5b8f3241b192b0b810122d4769/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:47:44.147221 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99c57b-2ac8-4ae8-88bd-2755f946e675 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:44.156474 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 795/1592] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:47:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:47:44.158998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da61be-c780-4be5-a4b3-f679d189c849 service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:44.168863 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da61be-c780-4be5-a4b3-f679d189c849 service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:44.245653 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99c57b-2ac8-4ae8-88bd-2755f946e675 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:44.251195 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44515467-bd5e-4f08-8eff-56fff1a360c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:44.252901 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44515467-bd5e-4f08-8eff-56fff1a360c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:44.253112 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44515467-bd5e-4f08-8eff-56fff1a360c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:44.253845 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44515467-bd5e-4f08-8eff-56fff1a360c1 admin admin] PATH_INFO: `/v3/projects/5c6ce84a298f4daa938864ec78d7a996/users/938083ff74d84224be142a26d8e17ad6/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:44.260603 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7851714-b45f-4e7e-a8f8-9256949632b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:44.261793 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7851714-b45f-4e7e-a8f8-9256949632b6 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:44.262368 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7851714-b45f-4e7e-a8f8-9256949632b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:44.262968 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7851714-b45f-4e7e-a8f8-9256949632b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:44.264336 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7851714-b45f-4e7e-a8f8-9256949632b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:44.266563 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7851714-b45f-4e7e-a8f8-9256949632b6 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:44.273322 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44515467-bd5e-4f08-8eff-56fff1a360c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c6ce84a298f4daa938864ec78d7a996, user_id=938083ff74d84224be142a26d8e17ad6, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:44.276238 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44515467-bd5e-4f08-8eff-56fff1a360c1 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:44.284695 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 796/1593] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:47:44.285754 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7851714-b45f-4e7e-a8f8-9256949632b6 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:44.299304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 798/1594] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:47:44 2022] PUT /identity/v3/projects/5c6ce84a298f4daa938864ec78d7a996/users/938083ff74d84224be142a26d8e17ad6/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:47:44.300608 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44515467-bd5e-4f08-8eff-56fff1a360c1 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:44.301747 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7851714-b45f-4e7e-a8f8-9256949632b6 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:44.369456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:44.382696 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f6be1-d8a6-4591-909a-8bc77d852227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:44.383627 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f6be1-d8a6-4591-909a-8bc77d852227 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:44.383849 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f6be1-d8a6-4591-909a-8bc77d852227 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:44.384198 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f6be1-d8a6-4591-909a-8bc77d852227 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:44.385190 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93f6be1-d8a6-4591-909a-8bc77d852227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:44.387129 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93f6be1-d8a6-4591-909a-8bc77d852227 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:44.402264 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 797/1595] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:47:44.402264 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f6be1-d8a6-4591-909a-8bc77d852227 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:44.413191 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44515467-bd5e-4f08-8eff-56fff1a360c1 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:44.414410 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f6be1-d8a6-4591-909a-8bc77d852227 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:44.419735 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef41d6d9-3110-428d-9874-1303798e2da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:44.420438 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41d6d9-3110-428d-9874-1303798e2da8 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:44.420690 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41d6d9-3110-428d-9874-1303798e2da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:44.421020 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41d6d9-3110-428d-9874-1303798e2da8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:44.421180 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e0504-440c-4af8-8dcb-34d73cf12a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:44.421759 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef41d6d9-3110-428d-9874-1303798e2da8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:44.422375 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e0504-440c-4af8-8dcb-34d73cf12a14 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:44.422840 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e0504-440c-4af8-8dcb-34d73cf12a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:44.423382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e0504-440c-4af8-8dcb-34d73cf12a14 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:44.423557 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef41d6d9-3110-428d-9874-1303798e2da8 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:44.424444 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7e0504-440c-4af8-8dcb-34d73cf12a14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:44.427028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7e0504-440c-4af8-8dcb-34d73cf12a14 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:44.435395 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 799/1596] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:47:44.436123 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef41d6d9-3110-428d-9874-1303798e2da8 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:44.443832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 798/1597] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:47:44.443832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20057128-0e1d-4bc0-960c-3a8d19e6bbc0 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:44.443832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20057128-0e1d-4bc0-960c-3a8d19e6bbc0 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:44.443832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20057128-0e1d-4bc0-960c-3a8d19e6bbc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:44.450992 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef41d6d9-3110-428d-9874-1303798e2da8 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:44.457605 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d4eac-5138-4092-a18e-474fa9da6a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:44.458297 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d4eac-5138-4092-a18e-474fa9da6a0b admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:44.458582 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d4eac-5138-4092-a18e-474fa9da6a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:44.458863 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d4eac-5138-4092-a18e-474fa9da6a0b admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:44.459641 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d4eac-5138-4092-a18e-474fa9da6a0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:44.461279 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d4eac-5138-4092-a18e-474fa9da6a0b admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:44.465659 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-20057128-0e1d-4bc0-960c-3a8d19e6bbc0 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:44.467079 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-20057128-0e1d-4bc0-960c-3a8d19e6bbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:44.472930 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 800/1598] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:47:44.473783 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d4eac-5138-4092-a18e-474fa9da6a0b admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:44.484150 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d4eac-5138-4092-a18e-474fa9da6a0b admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:44.485803 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 799/1599] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:44.487242 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014aadcb-a3ea-4e32-8514-853d1e05434c None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:44.487396 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014aadcb-a3ea-4e32-8514-853d1e05434c None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:44.487686 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014aadcb-a3ea-4e32-8514-853d1e05434c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:44.492994 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d780a5c-e4f8-4ed4-826e-174d4c0eabf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:44.494055 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d780a5c-e4f8-4ed4-826e-174d4c0eabf3 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:44.494901 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d780a5c-e4f8-4ed4-826e-174d4c0eabf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:44.495556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d780a5c-e4f8-4ed4-826e-174d4c0eabf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:44.496600 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d780a5c-e4f8-4ed4-826e-174d4c0eabf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:44.498631 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d780a5c-e4f8-4ed4-826e-174d4c0eabf3 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:44.500551 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:44.507843 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-014aadcb-a3ea-4e32-8514-853d1e05434c None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:44.509738 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-014aadcb-a3ea-4e32-8514-853d1e05434c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:44.515301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 801/1600] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:47:44.516463 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d780a5c-e4f8-4ed4-826e-174d4c0eabf3 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:44.526761 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 800/1601] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:44.528196 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014aadcb-a3ea-4e32-8514-853d1e05434c None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:44.534527 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d780a5c-e4f8-4ed4-826e-174d4c0eabf3 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:44.540158 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f7a49-0f06-4e18-85bb-a41661fd9c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:44.540840 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f7a49-0f06-4e18-85bb-a41661fd9c37 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:44.541115 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f7a49-0f06-4e18-85bb-a41661fd9c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:44.543208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f7a49-0f06-4e18-85bb-a41661fd9c37 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:44.543490 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014aadcb-a3ea-4e32-8514-853d1e05434c None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:44.543923 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f7a49-0f06-4e18-85bb-a41661fd9c37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:44.544199 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f7a49-0f06-4e18-85bb-a41661fd9c37 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:44.549364 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d76511b-9f12-45cb-a395-7ccfe4fe55fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:44.550349 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76511b-9f12-45cb-a395-7ccfe4fe55fb admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:44.550819 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76511b-9f12-45cb-a395-7ccfe4fe55fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:44.551117 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76511b-9f12-45cb-a395-7ccfe4fe55fb admin admin] PATH_INFO: `/v3/projects/b966dbac18614c979f674666ba817def/users/5f51124af1e846fb89552078c81afdb0/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:44.556999 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 802/1602] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:47:44.558203 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f7a49-0f06-4e18-85bb-a41661fd9c37 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:44.569719 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d76511b-9f12-45cb-a395-7ccfe4fe55fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b966dbac18614c979f674666ba817def, user_id=5f51124af1e846fb89552078c81afdb0, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:44.570622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: WARNING py.warnings [None req-6d76511b-9f12-45cb-a395-7ccfe4fe55fb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 14:47:44.570622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: warnings.warn(msg) Jul 20 14:47:44.570622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: Jul 20 14:47:44.572403 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f7a49-0f06-4e18-85bb-a41661fd9c37 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:44.572609 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d76511b-9f12-45cb-a395-7ccfe4fe55fb admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:44.581214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb87831-b6a0-44c5-bff5-d4a25860cfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:44.582172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb87831-b6a0-44c5-bff5-d4a25860cfd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:44.582455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb87831-b6a0-44c5-bff5-d4a25860cfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:44.582724 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb87831-b6a0-44c5-bff5-d4a25860cfd1 admin admin] PATH_INFO: `/v3/projects/7db1a5ab36714ee2990b9c6c10ba0aea/users/ee5a234904e64eb9b2faad85909d0507/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:44.601576 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 801/1603] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:47:44 2022] PUT /identity/v3/projects/b966dbac18614c979f674666ba817def/users/5f51124af1e846fb89552078c81afdb0/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:47:44.606846 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d76511b-9f12-45cb-a395-7ccfe4fe55fb admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:44.617454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb87831-b6a0-44c5-bff5-d4a25860cfd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7db1a5ab36714ee2990b9c6c10ba0aea, user_id=ee5a234904e64eb9b2faad85909d0507, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:44.620498 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb87831-b6a0-44c5-bff5-d4a25860cfd1 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:44.645246 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 803/1604] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:47:44 2022] PUT /identity/v3/projects/7db1a5ab36714ee2990b9c6c10ba0aea/users/ee5a234904e64eb9b2faad85909d0507/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:47:44.647394 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb87831-b6a0-44c5-bff5-d4a25860cfd1 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:44.700488 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d76511b-9f12-45cb-a395-7ccfe4fe55fb admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:44.762434 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb87831-b6a0-44c5-bff5-d4a25860cfd1 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:44.768505 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26afb9-4920-4033-8a0a-a6454e3b7019 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:44.769531 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26afb9-4920-4033-8a0a-a6454e3b7019 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:44.769997 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26afb9-4920-4033-8a0a-a6454e3b7019 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:44.770469 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26afb9-4920-4033-8a0a-a6454e3b7019 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:44.783939 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accea6ab-6582-4894-9495-f3ca9a8ee210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:44.785391 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accea6ab-6582-4894-9495-f3ca9a8ee210 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:44.786297 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accea6ab-6582-4894-9495-f3ca9a8ee210 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:44.787248 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accea6ab-6582-4894-9495-f3ca9a8ee210 admin admin] PATH_INFO: `/v3/projects/fcced7799400408083859f56c7b9181a/users/489508ae831e4b66b68a894efbc4bcf5/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:44.822324 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accea6ab-6582-4894-9495-f3ca9a8ee210 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcced7799400408083859f56c7b9181a, user_id=489508ae831e4b66b68a894efbc4bcf5, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:44.827960 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accea6ab-6582-4894-9495-f3ca9a8ee210 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:44.854575 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 802/1605] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:47:44 2022] PUT /identity/v3/projects/fcced7799400408083859f56c7b9181a/users/489508ae831e4b66b68a894efbc4bcf5/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:47:44.866026 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accea6ab-6582-4894-9495-f3ca9a8ee210 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:44.875156 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca26afb9-4920-4033-8a0a-a6454e3b7019 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:44.876829 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca26afb9-4920-4033-8a0a-a6454e3b7019 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:44.888924 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 804/1606] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:47:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:47:44.890079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26afb9-4920-4033-8a0a-a6454e3b7019 service neutron] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:44.901977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26afb9-4920-4033-8a0a-a6454e3b7019 service neutron] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:44.902470 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:44.946743 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accea6ab-6582-4894-9495-f3ca9a8ee210 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:44.954235 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d0c64-ce87-4233-a481-8e6a74aed114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:44.956115 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d0c64-ce87-4233-a481-8e6a74aed114 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:44.956438 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d0c64-ce87-4233-a481-8e6a74aed114 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:44.956764 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d0c64-ce87-4233-a481-8e6a74aed114 admin admin] PATH_INFO: `/v3/projects/ad412de7ac43432593c0bddd2e1c426a/users/9f3b85f9d80446d9a4ad9674fc465879/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:44.967061 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6ae7f-fb26-4eb7-b36b-85679b4a5fa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:44.968009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6ae7f-fb26-4eb7-b36b-85679b4a5fa9 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:44.968372 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6ae7f-fb26-4eb7-b36b-85679b4a5fa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:44.968730 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6ae7f-fb26-4eb7-b36b-85679b4a5fa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:44.975282 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7d0c64-ce87-4233-a481-8e6a74aed114 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad412de7ac43432593c0bddd2e1c426a, user_id=9f3b85f9d80446d9a4ad9674fc465879, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:44.978077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7d0c64-ce87-4233-a481-8e6a74aed114 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:44.981876 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e6ae7f-fb26-4eb7-b36b-85679b4a5fa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:44.985324 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e6ae7f-fb26-4eb7-b36b-85679b4a5fa9 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:44.993236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 803/1607] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:47:44 2022] PUT /identity/v3/projects/ad412de7ac43432593c0bddd2e1c426a/users/9f3b85f9d80446d9a4ad9674fc465879/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:47:44.994222 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d0c64-ce87-4233-a481-8e6a74aed114 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:45.009873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 805/1608] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:47:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:47:45.011323 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590ade0-8e77-43ac-8eac-d3918d32e51f None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:45.011602 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590ade0-8e77-43ac-8eac-d3918d32e51f None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:45.011942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590ade0-8e77-43ac-8eac-d3918d32e51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:45.031764 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-1590ade0-8e77-43ac-8eac-d3918d32e51f None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:45.032736 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-1590ade0-8e77-43ac-8eac-d3918d32e51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:45.043840 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:45.106309 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d0c64-ce87-4233-a481-8e6a74aed114 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:45.112614 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae6a0ff-f7e9-4f89-849a-c09b54632994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:45.113534 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6a0ff-f7e9-4f89-849a-c09b54632994 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:45.113814 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6a0ff-f7e9-4f89-849a-c09b54632994 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:45.114086 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6a0ff-f7e9-4f89-849a-c09b54632994 admin admin] PATH_INFO: `/v3/projects/70bb0d5b8f4440e9b5662cb4a7cbaad9/users/99097185f6744d778063374e18c9dca2/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:45.135001 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae6a0ff-f7e9-4f89-849a-c09b54632994 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70bb0d5b8f4440e9b5662cb4a7cbaad9, user_id=99097185f6744d778063374e18c9dca2, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:45.137734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae6a0ff-f7e9-4f89-849a-c09b54632994 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:45.141001 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 806/1609] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:45.142621 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8721d0-98c0-4e9d-bb7f-f166e6daac5c None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:45.142920 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8721d0-98c0-4e9d-bb7f-f166e6daac5c None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:45.143282 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8721d0-98c0-4e9d-bb7f-f166e6daac5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:45.163086 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 804/1610] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:47:44 2022] PUT /identity/v3/projects/70bb0d5b8f4440e9b5662cb4a7cbaad9/users/99097185f6744d778063374e18c9dca2/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:47:45.165361 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92b83b-0385-4a8c-bed2-b9dd19a35a39 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:45.165654 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92b83b-0385-4a8c-bed2-b9dd19a35a39 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:45.165939 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92b83b-0385-4a8c-bed2-b9dd19a35a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:45.172186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-df8721d0-98c0-4e9d-bb7f-f166e6daac5c None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:45.175109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-df8721d0-98c0-4e9d-bb7f-f166e6daac5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:45.222164 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-4a92b83b-0385-4a8c-bed2-b9dd19a35a39 None None] MFA Rules not processed for user `489508ae831e4b66b68a894efbc4bcf5`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:45.223352 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-4a92b83b-0385-4a8c-bed2-b9dd19a35a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:45.258089 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 807/1611] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:45.259803 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa78c2-d108-46a0-94da-589c65bcdbec None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:45.260241 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa78c2-d108-46a0-94da-589c65bcdbec None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:45.260653 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa78c2-d108-46a0-94da-589c65bcdbec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:45.309195 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-23fa78c2-d108-46a0-94da-589c65bcdbec None None] MFA Rules not processed for user `9f3b85f9d80446d9a4ad9674fc465879`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:45.309807 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 805/1612] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:45.310847 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92b83b-0385-4a8c-bed2-b9dd19a35a39 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:45.314347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-23fa78c2-d108-46a0-94da-589c65bcdbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:45.367177 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:45.432075 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92b83b-0385-4a8c-bed2-b9dd19a35a39 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:45.437598 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b4211-8ae0-426a-a325-a7f0ff8453ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:45.437864 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b4211-8ae0-426a-a325-a7f0ff8453ad service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:45.438152 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b4211-8ae0-426a-a325-a7f0ff8453ad service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:45.438440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b4211-8ae0-426a-a325-a7f0ff8453ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:45.446223 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 808/1613] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:45 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:45.448524 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd08ec8-c4dd-4675-bc90-d329041a369f None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:45.448818 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd08ec8-c4dd-4675-bc90-d329041a369f None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:45.449096 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd08ec8-c4dd-4675-bc90-d329041a369f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:45.449356 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97b4211-8ae0-426a-a325-a7f0ff8453ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:45.460038 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97b4211-8ae0-426a-a325-a7f0ff8453ad service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:45.472752 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 806/1614] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:47:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:47:45.473613 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b4211-8ae0-426a-a325-a7f0ff8453ad service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:45.492114 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-efd08ec8-c4dd-4675-bc90-d329041a369f None None] MFA Rules not processed for user `99097185f6744d778063374e18c9dca2`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:45.492375 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b4211-8ae0-426a-a325-a7f0ff8453ad service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:45.494049 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-efd08ec8-c4dd-4675-bc90-d329041a369f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:45.496510 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f37dac-7ddc-4c34-8e83-792aa36f6621 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:45.497194 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f37dac-7ddc-4c34-8e83-792aa36f6621 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:45.497479 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f37dac-7ddc-4c34-8e83-792aa36f6621 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:45.497758 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f37dac-7ddc-4c34-8e83-792aa36f6621 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:45.511494 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f37dac-7ddc-4c34-8e83-792aa36f6621 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:45.512996 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f37dac-7ddc-4c34-8e83-792aa36f6621 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:45.525149 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 807/1615] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:47:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:47:45.526706 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f37dac-7ddc-4c34-8e83-792aa36f6621 service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:45.537778 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f37dac-7ddc-4c34-8e83-792aa36f6621 service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:45.543406 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6295dd0-39c9-48c0-bf3e-ec43d2a427b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:45.544086 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6295dd0-39c9-48c0-bf3e-ec43d2a427b0 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:45.544641 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6295dd0-39c9-48c0-bf3e-ec43d2a427b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:45.544641 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6295dd0-39c9-48c0-bf3e-ec43d2a427b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:45.545344 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6295dd0-39c9-48c0-bf3e-ec43d2a427b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:45.547949 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6295dd0-39c9-48c0-bf3e-ec43d2a427b0 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:45.580237 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:45.591790 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 808/1616] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:47:45 2022] POST /identity/v3/projects => generated 393 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:47:45.595992 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6295dd0-39c9-48c0-bf3e-ec43d2a427b0 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:45.664771 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 809/1617] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:45.665534 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd08ec8-c4dd-4675-bc90-d329041a369f None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:45.689812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd08ec8-c4dd-4675-bc90-d329041a369f None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:45.732103 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6295dd0-39c9-48c0-bf3e-ec43d2a427b0 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:45.744027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7e081-5ba3-49cd-b0fe-590b2683fe5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:45.744555 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7e081-5ba3-49cd-b0fe-590b2683fe5c service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:45.745537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7e081-5ba3-49cd-b0fe-590b2683fe5c service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:45.746338 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7e081-5ba3-49cd-b0fe-590b2683fe5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:45.788107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f315cb-ad0d-4aaf-96b8-ee0782821a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:45.789477 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f315cb-ad0d-4aaf-96b8-ee0782821a80 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:45.790415 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f315cb-ad0d-4aaf-96b8-ee0782821a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:45.790867 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f315cb-ad0d-4aaf-96b8-ee0782821a80 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:45.792287 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f315cb-ad0d-4aaf-96b8-ee0782821a80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:45.796943 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f315cb-ad0d-4aaf-96b8-ee0782821a80 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:45.848390 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 810/1618] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:47:45 2022] POST /identity/v3/users => generated 303 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:47:45.849602 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30ebb4-6e81-4f3c-90e0-a8579ecc4a09 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:45.850035 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30ebb4-6e81-4f3c-90e0-a8579ecc4a09 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:45.850626 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30ebb4-6e81-4f3c-90e0-a8579ecc4a09 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:45.855030 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 811/1619] 10.222.0.15 () {54 vars in 962 bytes} [Wed Jul 20 14:47:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:47:45.856456 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30ebb4-6e81-4f3c-90e0-a8579ecc4a09 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:45.879066 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7e081-5ba3-49cd-b0fe-590b2683fe5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:45.880680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7e081-5ba3-49cd-b0fe-590b2683fe5c service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:45.884471 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30ebb4-6e81-4f3c-90e0-a8579ecc4a09 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:45.895852 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf060f07-1565-4e5a-a8c2-d1ea859f2f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:45.896603 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf060f07-1565-4e5a-a8c2-d1ea859f2f4b admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:45.896893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf060f07-1565-4e5a-a8c2-d1ea859f2f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:45.897225 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf060f07-1565-4e5a-a8c2-d1ea859f2f4b admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:45.901579 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf060f07-1565-4e5a-a8c2-d1ea859f2f4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:45.903396 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 809/1620] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:47:45 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:47:45.907378 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf060f07-1565-4e5a-a8c2-d1ea859f2f4b admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:45.908271 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a815a96-a45a-4aaf-b3e1-2f3e08da7b1d None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:45.908668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a815a96-a45a-4aaf-b3e1-2f3e08da7b1d None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:45.908930 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a815a96-a45a-4aaf-b3e1-2f3e08da7b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:45.954963 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 812/1621] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:47:45.959670 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf060f07-1565-4e5a-a8c2-d1ea859f2f4b admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:45.984121 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-9a815a96-a45a-4aaf-b3e1-2f3e08da7b1d None None] MFA Rules not processed for user `5d4a107ed04348b59d20d1703dbe04eb`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:45.985760 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-9a815a96-a45a-4aaf-b3e1-2f3e08da7b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:45.990625 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf060f07-1565-4e5a-a8c2-d1ea859f2f4b admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:46.001204 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c6d03-90aa-44e0-86a8-4b2a8ab2fd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:46.005310 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c6d03-90aa-44e0-86a8-4b2a8ab2fd40 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:46.005636 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c6d03-90aa-44e0-86a8-4b2a8ab2fd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:46.007159 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c6d03-90aa-44e0-86a8-4b2a8ab2fd40 admin admin] PATH_INFO: `/v3/projects/3b92a2c7f66d470bbfb0db1da9b609bb/users/288d0774b20249ffb4c07535b0d8d349/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:46.026009 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:46.044311 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2c6d03-90aa-44e0-86a8-4b2a8ab2fd40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b92a2c7f66d470bbfb0db1da9b609bb, user_id=288d0774b20249ffb4c07535b0d8d349, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:46.051483 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2c6d03-90aa-44e0-86a8-4b2a8ab2fd40 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:46.088804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 813/1622] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:47:45 2022] PUT /identity/v3/projects/3b92a2c7f66d470bbfb0db1da9b609bb/users/288d0774b20249ffb4c07535b0d8d349/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:47:46.092566 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c6d03-90aa-44e0-86a8-4b2a8ab2fd40 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:46.121995 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 810/1623] 10.222.0.15 () {58 vars in 1059 bytes} [Wed Jul 20 14:47:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:46.142385 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a815a96-a45a-4aaf-b3e1-2f3e08da7b1d None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:46.156597 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a815a96-a45a-4aaf-b3e1-2f3e08da7b1d None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:46.213946 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c6d03-90aa-44e0-86a8-4b2a8ab2fd40 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:46.223681 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eafbb2a-b811-4659-a7e5-cee3bfb1b13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:46.224537 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafbb2a-b811-4659-a7e5-cee3bfb1b13d admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:46.224824 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafbb2a-b811-4659-a7e5-cee3bfb1b13d admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:46.226502 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafbb2a-b811-4659-a7e5-cee3bfb1b13d admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:46.231165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eafbb2a-b811-4659-a7e5-cee3bfb1b13d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:46.235611 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eafbb2a-b811-4659-a7e5-cee3bfb1b13d admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:46.267824 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 814/1624] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:47:46 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:47:46.268783 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eafbb2a-b811-4659-a7e5-cee3bfb1b13d admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:46.292608 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eafbb2a-b811-4659-a7e5-cee3bfb1b13d admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:46.304004 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab842837-87e6-43ef-9400-86db30791a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:46.304939 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab842837-87e6-43ef-9400-86db30791a45 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:46.308595 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab842837-87e6-43ef-9400-86db30791a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:46.308969 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab842837-87e6-43ef-9400-86db30791a45 admin admin] PATH_INFO: `/v3/projects/3b92a2c7f66d470bbfb0db1da9b609bb/users/288d0774b20249ffb4c07535b0d8d349/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:46.314947 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab842837-87e6-43ef-9400-86db30791a45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b92a2c7f66d470bbfb0db1da9b609bb, user_id=288d0774b20249ffb4c07535b0d8d349, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:46.321965 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab842837-87e6-43ef-9400-86db30791a45 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:46.349366 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c0fe8-b55e-4b08-8f55-54e8f9818276 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:46.350781 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c0fe8-b55e-4b08-8f55-54e8f9818276 service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:46.351920 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 815/1625] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:47:46 2022] PUT /identity/v3/projects/3b92a2c7f66d470bbfb0db1da9b609bb/users/288d0774b20249ffb4c07535b0d8d349/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:47:46.352045 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c0fe8-b55e-4b08-8f55-54e8f9818276 service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:46.362137 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c0fe8-b55e-4b08-8f55-54e8f9818276 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:46.368351 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03174995-119f-4d36-a0de-1c5cc06b0ddc None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:46.368739 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03174995-119f-4d36-a0de-1c5cc06b0ddc None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:46.369024 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03174995-119f-4d36-a0de-1c5cc06b0ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:46.427323 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:46.449049 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-03174995-119f-4d36-a0de-1c5cc06b0ddc None None] MFA Rules not processed for user `288d0774b20249ffb4c07535b0d8d349`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:46.450383 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-03174995-119f-4d36-a0de-1c5cc06b0ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:46.531248 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3c0fe8-b55e-4b08-8f55-54e8f9818276 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:46.537342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3c0fe8-b55e-4b08-8f55-54e8f9818276 service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:46.567806 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 811/1626] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:47:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 14:47:46.611144 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 816/1627] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:46 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:46.625591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c0fe8-b55e-4b08-8f55-54e8f9818276 service nova] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:46.785375 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c0fe8-b55e-4b08-8f55-54e8f9818276 service nova] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:46.812227 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:46.894390 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2f1b0-8600-49a8-b62a-43d823b7a5ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:46.913073 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2f1b0-8600-49a8-b62a-43d823b7a5ea service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:46.914241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2f1b0-8600-49a8-b62a-43d823b7a5ea service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:46.915297 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2f1b0-8600-49a8-b62a-43d823b7a5ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:46.944125 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d2f1b0-8600-49a8-b62a-43d823b7a5ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:46.946478 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d2f1b0-8600-49a8-b62a-43d823b7a5ea service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:46.976970 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 812/1628] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:47:46 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 14:47:47.189666 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03174995-119f-4d36-a0de-1c5cc06b0ddc None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:47.209910 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03174995-119f-4d36-a0de-1c5cc06b0ddc None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:47.214896 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921688f0-3436-4b05-81ac-283d1dec4ec7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:47.216036 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921688f0-3436-4b05-81ac-283d1dec4ec7 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:47.216148 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921688f0-3436-4b05-81ac-283d1dec4ec7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:47.216616 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921688f0-3436-4b05-81ac-283d1dec4ec7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:47.303882 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921688f0-3436-4b05-81ac-283d1dec4ec7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:47.306361 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921688f0-3436-4b05-81ac-283d1dec4ec7 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:47.329272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 817/1629] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:47:47 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:47:47.793212 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435bc874-3734-40ca-b1c4-27b6acc47970 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:47.793772 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435bc874-3734-40ca-b1c4-27b6acc47970 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:47.793772 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435bc874-3734-40ca-b1c4-27b6acc47970 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:47.795555 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 813/1630] 10.222.0.15 () {54 vars in 962 bytes} [Wed Jul 20 14:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:47:47.883441 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec012a-34bb-4947-8c17-986235bf309d None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:47.883441 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec012a-34bb-4947-8c17-986235bf309d None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:47.883441 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec012a-34bb-4947-8c17-986235bf309d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:47.883441 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-b5ec012a-34bb-4947-8c17-986235bf309d None None] MFA Rules not processed for user `5d4a107ed04348b59d20d1703dbe04eb`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:47.883441 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-b5ec012a-34bb-4947-8c17-986235bf309d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:48.044548 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387bcf4d-cd6c-4692-a727-54707e86d19b None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:48.045212 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387bcf4d-cd6c-4692-a727-54707e86d19b None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:48.045713 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387bcf4d-cd6c-4692-a727-54707e86d19b None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:48.046698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 818/1631] 10.222.0.15 () {58 vars in 1059 bytes} [Wed Jul 20 14:47:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:47:48.048957 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 814/1632] 10.222.0.15 () {54 vars in 912 bytes} [Wed Jul 20 14:47:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 14:47:48.055302 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6beab-4d39-4eba-84dc-013c75cf9d6a None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:48.056669 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6beab-4d39-4eba-84dc-013c75cf9d6a None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:48.056963 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6beab-4d39-4eba-84dc-013c75cf9d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:48.102889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-f6b6beab-4d39-4eba-84dc-013c75cf9d6a None None] MFA Rules not processed for user `5d4a107ed04348b59d20d1703dbe04eb`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:48.107634 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-f6b6beab-4d39-4eba-84dc-013c75cf9d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:48.141017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 819/1633] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Jul 20 14:47:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:48.169953 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387bcf4d-cd6c-4692-a727-54707e86d19b None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:48.185947 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387bcf4d-cd6c-4692-a727-54707e86d19b None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:48.190404 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4cf072-c3a5-4abb-93b9-efb7beb65026 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:48.191446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cf072-c3a5-4abb-93b9-efb7beb65026 service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:48.191902 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cf072-c3a5-4abb-93b9-efb7beb65026 service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:48.192342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cf072-c3a5-4abb-93b9-efb7beb65026 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:48.212460 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4cf072-c3a5-4abb-93b9-efb7beb65026 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:48.214895 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4cf072-c3a5-4abb-93b9-efb7beb65026 service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:48.257888 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 815/1634] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:47:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:47:48.371575 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed95168-7074-4492-98b6-6c179b7c3499 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:48.371908 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed95168-7074-4492-98b6-6c179b7c3499 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:48.372742 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed95168-7074-4492-98b6-6c179b7c3499 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:48.374985 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 820/1635] 10.222.0.15 () {54 vars in 912 bytes} [Wed Jul 20 14:47:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:47:48.386838 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f650aa-c6f4-4c95-83a2-0a4b3ae80d33 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:48.388067 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f650aa-c6f4-4c95-83a2-0a4b3ae80d33 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:48.389303 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f650aa-c6f4-4c95-83a2-0a4b3ae80d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:48.453217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-a5f650aa-c6f4-4c95-83a2-0a4b3ae80d33 None None] MFA Rules not processed for user `5d4a107ed04348b59d20d1703dbe04eb`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:48.454533 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-a5f650aa-c6f4-4c95-83a2-0a4b3ae80d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:48.468731 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 816/1636] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Jul 20 14:47:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:48.481968 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed95168-7074-4492-98b6-6c179b7c3499 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:48.499184 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed95168-7074-4492-98b6-6c179b7c3499 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:48.503920 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171015d7-9265-49bc-b0af-d1ba3b535f5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:48.504971 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171015d7-9265-49bc-b0af-d1ba3b535f5a service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:48.505433 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171015d7-9265-49bc-b0af-d1ba3b535f5a service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:48.505888 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171015d7-9265-49bc-b0af-d1ba3b535f5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:48.526771 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171015d7-9265-49bc-b0af-d1ba3b535f5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:48.529044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171015d7-9265-49bc-b0af-d1ba3b535f5a service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:48.543450 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 821/1637] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:47:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:47:48.703533 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f650aa-c6f4-4c95-83a2-0a4b3ae80d33 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:48.843563 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f650aa-c6f4-4c95-83a2-0a4b3ae80d33 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:48.849866 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439228e-2cc8-4940-8d09-ea3322c88ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:48.851184 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439228e-2cc8-4940-8d09-ea3322c88ce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:48.851787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439228e-2cc8-4940-8d09-ea3322c88ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:48.852304 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439228e-2cc8-4940-8d09-ea3322c88ce5 admin admin] PATH_INFO: `/v3/users/99097185f6744d778063374e18c9dca2` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:48.853969 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b439228e-2cc8-4940-8d09-ea3322c88ce5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99097185f6744d778063374e18c9dca2)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:48.855858 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: WARNING py.warnings [None req-b439228e-2cc8-4940-8d09-ea3322c88ce5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 14:47:48.855858 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: warnings.warn(msg) Jul 20 14:47:48.855858 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: Jul 20 14:47:48.858363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b439228e-2cc8-4940-8d09-ea3322c88ce5 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:48.963917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-b439228e-2cc8-4940-8d09-ea3322c88ce5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99097185f6744d778063374e18c9dca2'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:47:49.004902 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-b439228e-2cc8-4940-8d09-ea3322c88ce5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99097185f6744d778063374e18c9dca2'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:47:49.006251 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-b439228e-2cc8-4940-8d09-ea3322c88ce5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99097185f6744d778063374e18c9dca2'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:47:49.007203 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-b439228e-2cc8-4940-8d09-ea3322c88ce5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99097185f6744d778063374e18c9dca2'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:47:49.038130 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-b439228e-2cc8-4940-8d09-ea3322c88ce5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99097185f6744d778063374e18c9dca2'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:47:49.060463 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 817/1638] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:47:48 2022] DELETE /identity/v3/users/99097185f6744d778063374e18c9dca2 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:47:49.065137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff62ae1-375c-40ef-8ad8-3c2731c9d372 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:49.066984 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff62ae1-375c-40ef-8ad8-3c2731c9d372 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:49.067307 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff62ae1-375c-40ef-8ad8-3c2731c9d372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:49.082073 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:49.089792 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-6ff62ae1-375c-40ef-8ad8-3c2731c9d372 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:49.091320 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-6ff62ae1-375c-40ef-8ad8-3c2731c9d372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:49.249636 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 822/1639] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:49.273024 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439228e-2cc8-4940-8d09-ea3322c88ce5 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:49.275412 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-b439228e-2cc8-4940-8d09-ea3322c88ce5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:49.353282 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:49.483928 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439228e-2cc8-4940-8d09-ea3322c88ce5 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:49.497652 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4493ed0-06af-42b6-8895-99ab0dc683d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:49.500080 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4493ed0-06af-42b6-8895-99ab0dc683d9 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:49.500371 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4493ed0-06af-42b6-8895-99ab0dc683d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:49.500663 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4493ed0-06af-42b6-8895-99ab0dc683d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:49.524544 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4493ed0-06af-42b6-8895-99ab0dc683d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:49.526552 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4493ed0-06af-42b6-8895-99ab0dc683d9 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:49.553852 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 818/1640] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:47:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:47:51.471870 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff62ae1-375c-40ef-8ad8-3c2731c9d372 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:51.473493 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-6ff62ae1-375c-40ef-8ad8-3c2731c9d372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:51.501787 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff62ae1-375c-40ef-8ad8-3c2731c9d372 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:51.520754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e6f13-f37f-438f-b14d-be917fc7a713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:51.521724 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e6f13-f37f-438f-b14d-be917fc7a713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:51.522202 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e6f13-f37f-438f-b14d-be917fc7a713 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:51.522649 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e6f13-f37f-438f-b14d-be917fc7a713 admin admin] PATH_INFO: `/v3/projects/70bb0d5b8f4440e9b5662cb4a7cbaad9` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:51.524472 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721e6f13-f37f-438f-b14d-be917fc7a713 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70bb0d5b8f4440e9b5662cb4a7cbaad9)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:51.525832 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: WARNING py.warnings [None req-721e6f13-f37f-438f-b14d-be917fc7a713 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 14:47:51.525832 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: warnings.warn(msg) Jul 20 14:47:51.525832 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: Jul 20 14:47:51.532159 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721e6f13-f37f-438f-b14d-be917fc7a713 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:51.733659 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-721e6f13-f37f-438f-b14d-be917fc7a713 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70bb0d5b8f4440e9b5662cb4a7cbaad9', 'request_id': 'req-721e6f13-f37f-438f-b14d-be917fc7a713'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:47:51.760517 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-721e6f13-f37f-438f-b14d-be917fc7a713 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70bb0d5b8f4440e9b5662cb4a7cbaad9', 'request_id': 'req-721e6f13-f37f-438f-b14d-be917fc7a713'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:47:51.774657 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-721e6f13-f37f-438f-b14d-be917fc7a713 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70bb0d5b8f4440e9b5662cb4a7cbaad9', 'request_id': 'req-721e6f13-f37f-438f-b14d-be917fc7a713'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:47:51.866199 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-721e6f13-f37f-438f-b14d-be917fc7a713 admin admin] The token cache is being invalidate because project 70bb0d5b8f4440e9b5662cb4a7cbaad9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:47:51.866767 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-721e6f13-f37f-438f-b14d-be917fc7a713 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:47:51.867981 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-721e6f13-f37f-438f-b14d-be917fc7a713 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:47:51.870202 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 823/1641] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:47:51 2022] DELETE /identity/v3/projects/70bb0d5b8f4440e9b5662cb4a7cbaad9 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:47:51.875849 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1762ac-9b02-4e36-bfc1-a947026e786d None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:51.875849 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1762ac-9b02-4e36-bfc1-a947026e786d None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:51.876732 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1762ac-9b02-4e36-bfc1-a947026e786d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:51.923938 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-ef1762ac-9b02-4e36-bfc1-a947026e786d None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:51.925198 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-ef1762ac-9b02-4e36-bfc1-a947026e786d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:52.043762 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 819/1642] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:52.051389 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd87cac-cb6f-4a69-8319-f83980963793 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:52.051952 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd87cac-cb6f-4a69-8319-f83980963793 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:52.052444 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd87cac-cb6f-4a69-8319-f83980963793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:52.103622 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-0bd87cac-cb6f-4a69-8319-f83980963793 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:52.106835 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-0bd87cac-cb6f-4a69-8319-f83980963793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:52.138273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 824/1643] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:52.142023 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1762ac-9b02-4e36-bfc1-a947026e786d None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:52.157366 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1762ac-9b02-4e36-bfc1-a947026e786d None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:52.174573 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5429b0d-bae7-44d7-8c75-a82d4afee6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:52.175340 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5429b0d-bae7-44d7-8c75-a82d4afee6b4 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:52.175632 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5429b0d-bae7-44d7-8c75-a82d4afee6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:52.176210 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5429b0d-bae7-44d7-8c75-a82d4afee6b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:52.178224 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5429b0d-bae7-44d7-8c75-a82d4afee6b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:47:52.178553 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5429b0d-bae7-44d7-8c75-a82d4afee6b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:52.188317 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5429b0d-bae7-44d7-8c75-a82d4afee6b4 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:52.225471 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 820/1644] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:47:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:47:52.229676 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd87cac-cb6f-4a69-8319-f83980963793 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:52.242469 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd87cac-cb6f-4a69-8319-f83980963793 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:52.255278 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc20a7-bb2e-44e6-951e-fefd61a16fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:52.257119 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc20a7-bb2e-44e6-951e-fefd61a16fbc admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:52.257882 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc20a7-bb2e-44e6-951e-fefd61a16fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:52.259816 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc20a7-bb2e-44e6-951e-fefd61a16fbc admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:52.262032 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc20a7-bb2e-44e6-951e-fefd61a16fbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:52.268644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc20a7-bb2e-44e6-951e-fefd61a16fbc admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:52.302073 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5429b0d-bae7-44d7-8c75-a82d4afee6b4 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:52.307296 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-b5429b0d-bae7-44d7-8c75-a82d4afee6b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:52.335938 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5429b0d-bae7-44d7-8c75-a82d4afee6b4 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:52.338528 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 825/1645] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:47:52 2022] POST /identity/v3/projects => generated 371 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:47:52.342775 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc20a7-bb2e-44e6-951e-fefd61a16fbc admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:52.506330 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f4eef-0376-4ff1-9b8d-ea9bd3b1c2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:52.507482 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f4eef-0376-4ff1-9b8d-ea9bd3b1c2e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:52.507781 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f4eef-0376-4ff1-9b8d-ea9bd3b1c2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:52.508065 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f4eef-0376-4ff1-9b8d-ea9bd3b1c2e8 admin admin] PATH_INFO: `/v3/users/489508ae831e4b66b68a894efbc4bcf5` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:52.511354 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f4eef-0376-4ff1-9b8d-ea9bd3b1c2e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=489508ae831e4b66b68a894efbc4bcf5)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:52.516358 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f4eef-0376-4ff1-9b8d-ea9bd3b1c2e8 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:52.520259 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc20a7-bb2e-44e6-951e-fefd61a16fbc admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:52.535947 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2aef43-e32e-4c6e-9bab-1b50c02017bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:52.536660 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2aef43-e32e-4c6e-9bab-1b50c02017bf admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:52.536950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2aef43-e32e-4c6e-9bab-1b50c02017bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:52.537234 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2aef43-e32e-4c6e-9bab-1b50c02017bf admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:52.540332 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2aef43-e32e-4c6e-9bab-1b50c02017bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:52.542375 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2aef43-e32e-4c6e-9bab-1b50c02017bf admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:52.567519 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:52.606249 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 826/1646] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:47:52 2022] POST /identity/v3/users => generated 286 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:47:52.610754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2aef43-e32e-4c6e-9bab-1b50c02017bf admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:52.641705 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2aef43-e32e-4c6e-9bab-1b50c02017bf admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:52.645841 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-861f4eef-0376-4ff1-9b8d-ea9bd3b1c2e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '489508ae831e4b66b68a894efbc4bcf5'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:47:52.664196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c471546f-162d-44fb-a2ad-870fc3b5e8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:52.685570 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c471546f-162d-44fb-a2ad-870fc3b5e8b6 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:52.686227 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:52.688161 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c471546f-162d-44fb-a2ad-870fc3b5e8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:52.689765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c471546f-162d-44fb-a2ad-870fc3b5e8b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:52.695488 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c471546f-162d-44fb-a2ad-870fc3b5e8b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:52.701627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c471546f-162d-44fb-a2ad-870fc3b5e8b6 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:52.715351 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-861f4eef-0376-4ff1-9b8d-ea9bd3b1c2e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '489508ae831e4b66b68a894efbc4bcf5'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:47:52.716701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-861f4eef-0376-4ff1-9b8d-ea9bd3b1c2e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '489508ae831e4b66b68a894efbc4bcf5'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:47:52.717755 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-861f4eef-0376-4ff1-9b8d-ea9bd3b1c2e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '489508ae831e4b66b68a894efbc4bcf5'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:47:52.725490 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 827/1647] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:47:52.730265 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-861f4eef-0376-4ff1-9b8d-ea9bd3b1c2e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '489508ae831e4b66b68a894efbc4bcf5'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:47:52.730463 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c471546f-162d-44fb-a2ad-870fc3b5e8b6 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:52.735848 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-c471546f-162d-44fb-a2ad-870fc3b5e8b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:52.762293 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 821/1648] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:47:52 2022] DELETE /identity/v3/users/489508ae831e4b66b68a894efbc4bcf5 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 14:47:52.768251 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fe1cc-e5f3-4e51-bd3c-d191a916a3a0 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:52.769710 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fe1cc-e5f3-4e51-bd3c-d191a916a3a0 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:52.770260 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fe1cc-e5f3-4e51-bd3c-d191a916a3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:52.784167 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c471546f-162d-44fb-a2ad-870fc3b5e8b6 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:52.819442 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-b43fe1cc-e5f3-4e51-bd3c-d191a916a3a0 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:52.823238 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-b43fe1cc-e5f3-4e51-bd3c-d191a916a3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:52.951599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1739e-e6cf-4800-bb4a-0c51dd43a116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:52.952308 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1739e-e6cf-4800-bb4a-0c51dd43a116 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:52.952595 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1739e-e6cf-4800-bb4a-0c51dd43a116 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:52.953386 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1739e-e6cf-4800-bb4a-0c51dd43a116 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:52.954100 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c1739e-e6cf-4800-bb4a-0c51dd43a116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:52.955825 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c1739e-e6cf-4800-bb4a-0c51dd43a116 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:52.994392 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 828/1649] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:47:52.999801 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1739e-e6cf-4800-bb4a-0c51dd43a116 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:53.029390 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1739e-e6cf-4800-bb4a-0c51dd43a116 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:53.056572 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea0b62b-c0b1-498f-962c-a5c5d0b82025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:53.057514 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0b62b-c0b1-498f-962c-a5c5d0b82025 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:53.057817 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0b62b-c0b1-498f-962c-a5c5d0b82025 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:53.058090 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0b62b-c0b1-498f-962c-a5c5d0b82025 admin admin] PATH_INFO: `/v3/projects/dfe7c5ee1264469b88c7c4f43e7ef048/users/0432bf57115d453e88f561e29ea803f4/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:53.063794 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 822/1650] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:53.086635 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea0b62b-c0b1-498f-962c-a5c5d0b82025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfe7c5ee1264469b88c7c4f43e7ef048, user_id=0432bf57115d453e88f561e29ea803f4, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:53.089598 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea0b62b-c0b1-498f-962c-a5c5d0b82025 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:53.123097 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 829/1651] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:47:52 2022] PUT /identity/v3/projects/dfe7c5ee1264469b88c7c4f43e7ef048/users/0432bf57115d453e88f561e29ea803f4/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:47:53.135083 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e8ed5-9026-456a-a313-597f4971489b None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:53.136104 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e8ed5-9026-456a-a313-597f4971489b None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:53.136869 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e8ed5-9026-456a-a313-597f4971489b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:53.150071 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea0b62b-c0b1-498f-962c-a5c5d0b82025 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:53.152682 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-0ea0b62b-c0b1-498f-962c-a5c5d0b82025 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:53.202650 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-192e8ed5-9026-456a-a313-597f4971489b None None] MFA Rules not processed for user `0432bf57115d453e88f561e29ea803f4`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:53.204630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-192e8ed5-9026-456a-a313-597f4971489b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:53.280766 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea0b62b-c0b1-498f-962c-a5c5d0b82025 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:53.289092 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548b8dd-8635-4ce0-b762-f5fda54742c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:53.290013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548b8dd-8635-4ce0-b762-f5fda54742c7 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:53.290359 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548b8dd-8635-4ce0-b762-f5fda54742c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:53.293798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548b8dd-8635-4ce0-b762-f5fda54742c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:53.354819 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 823/1652] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:53.365867 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e8ed5-9026-456a-a313-597f4971489b None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:53.460316 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5548b8dd-8635-4ce0-b762-f5fda54742c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:53.462512 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5548b8dd-8635-4ce0-b762-f5fda54742c7 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:53.490867 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 830/1653] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:47:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 14:47:53.555242 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e8ed5-9026-456a-a313-597f4971489b None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:53.564810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e0c3e-91d7-4e23-9ead-f47e1f99b8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:53.565482 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e0c3e-91d7-4e23-9ead-f47e1f99b8ff admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:53.565767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e0c3e-91d7-4e23-9ead-f47e1f99b8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:53.566050 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e0c3e-91d7-4e23-9ead-f47e1f99b8ff admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:53.566783 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9e0c3e-91d7-4e23-9ead-f47e1f99b8ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:53.568869 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9e0c3e-91d7-4e23-9ead-f47e1f99b8ff admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:53.581613 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:53.658593 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 824/1654] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:47:53 2022] POST /identity/v3/projects => generated 373 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 14:47:53.662612 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548b8dd-8635-4ce0-b762-f5fda54742c7 service neutron] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:53.668503 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:53.682168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548b8dd-8635-4ce0-b762-f5fda54742c7 service neutron] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:53.900084 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd806291-4190-4d0b-ade2-0681dbfc1152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:53.900853 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd806291-4190-4d0b-ade2-0681dbfc1152 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:53.901140 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd806291-4190-4d0b-ade2-0681dbfc1152 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:53.901416 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd806291-4190-4d0b-ade2-0681dbfc1152 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:53.902642 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd806291-4190-4d0b-ade2-0681dbfc1152 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:53.905048 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd806291-4190-4d0b-ade2-0681dbfc1152 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:53.968125 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 831/1655] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:47:53 2022] POST /identity/v3/users => generated 293 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:47:53.975189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e0c3e-91d7-4e23-9ead-f47e1f99b8ff admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:54.045041 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e0c3e-91d7-4e23-9ead-f47e1f99b8ff admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:54.056419 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4a746-a65b-4aa4-9a04-f4687f9d30f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:54.057950 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4a746-a65b-4aa4-9a04-f4687f9d30f8 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:54.058488 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4a746-a65b-4aa4-9a04-f4687f9d30f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:54.058986 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4a746-a65b-4aa4-9a04-f4687f9d30f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:54.060371 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be4a746-a65b-4aa4-9a04-f4687f9d30f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:54.062516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be4a746-a65b-4aa4-9a04-f4687f9d30f8 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:54.095038 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 825/1656] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:47:54.098997 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd806291-4190-4d0b-ade2-0681dbfc1152 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:54.120522 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd806291-4190-4d0b-ade2-0681dbfc1152 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:54.136088 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6280aa48-3eac-4ebc-80a1-944069961484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:54.137756 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280aa48-3eac-4ebc-80a1-944069961484 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:54.138011 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280aa48-3eac-4ebc-80a1-944069961484 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:54.138397 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280aa48-3eac-4ebc-80a1-944069961484 admin admin] PATH_INFO: `/v3/projects/a19d5bde23624fbd9daab52c3c2c3d08/users/1729d01bb2284886852622a8b40f3424/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:54.185807 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6280aa48-3eac-4ebc-80a1-944069961484 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a19d5bde23624fbd9daab52c3c2c3d08, user_id=1729d01bb2284886852622a8b40f3424, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:54.188743 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6280aa48-3eac-4ebc-80a1-944069961484 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:54.228697 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 832/1657] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:47:54 2022] PUT /identity/v3/projects/a19d5bde23624fbd9daab52c3c2c3d08/users/1729d01bb2284886852622a8b40f3424/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:47:54.232756 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4a746-a65b-4aa4-9a04-f4687f9d30f8 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:54.259319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4a746-a65b-4aa4-9a04-f4687f9d30f8 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:54.461042 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b3acd2-dcd2-44cf-8c29-7f6b5006bfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:54.468959 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3acd2-dcd2-44cf-8c29-7f6b5006bfd0 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:54.474045 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3acd2-dcd2-44cf-8c29-7f6b5006bfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:54.477079 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3acd2-dcd2-44cf-8c29-7f6b5006bfd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:54.489398 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b3acd2-dcd2-44cf-8c29-7f6b5006bfd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:54.494238 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b3acd2-dcd2-44cf-8c29-7f6b5006bfd0 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:54.533608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 826/1658] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:47:54.537852 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6280aa48-3eac-4ebc-80a1-944069961484 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:54.558586 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6280aa48-3eac-4ebc-80a1-944069961484 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:54.600574 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69998f42-d54e-465a-b8b6-332734b46ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:54.601521 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69998f42-d54e-465a-b8b6-332734b46ed6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:54.602000 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69998f42-d54e-465a-b8b6-332734b46ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:54.602262 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69998f42-d54e-465a-b8b6-332734b46ed6 admin admin] PATH_INFO: `/v3/projects/a19d5bde23624fbd9daab52c3c2c3d08/users/1729d01bb2284886852622a8b40f3424/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:54.606144 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69998f42-d54e-465a-b8b6-332734b46ed6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a19d5bde23624fbd9daab52c3c2c3d08, user_id=1729d01bb2284886852622a8b40f3424, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:54.609907 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69998f42-d54e-465a-b8b6-332734b46ed6 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:54.663366 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 833/1659] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:47:54 2022] PUT /identity/v3/projects/a19d5bde23624fbd9daab52c3c2c3d08/users/1729d01bb2284886852622a8b40f3424/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:47:54.675072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e352f8-8754-4527-8dd4-5685cbfc37ef None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:54.675751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e352f8-8754-4527-8dd4-5685cbfc37ef None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:54.676212 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e352f8-8754-4527-8dd4-5685cbfc37ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:54.737145 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-f9e352f8-8754-4527-8dd4-5685cbfc37ef None None] MFA Rules not processed for user `1729d01bb2284886852622a8b40f3424`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:54.738965 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-f9e352f8-8754-4527-8dd4-5685cbfc37ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:54.904784 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 827/1660] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:54 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:54.919110 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69998f42-d54e-465a-b8b6-332734b46ed6 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:54.922095 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-69998f42-d54e-465a-b8b6-332734b46ed6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:55.065567 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69998f42-d54e-465a-b8b6-332734b46ed6 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:55.075509 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658cbd5-f941-42cf-963f-cfb7b5a19ac7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:55.076204 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658cbd5-f941-42cf-963f-cfb7b5a19ac7 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:55.076489 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658cbd5-f941-42cf-963f-cfb7b5a19ac7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:55.076770 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658cbd5-f941-42cf-963f-cfb7b5a19ac7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:55.102623 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658cbd5-f941-42cf-963f-cfb7b5a19ac7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:55.104067 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658cbd5-f941-42cf-963f-cfb7b5a19ac7 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:55.136408 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 834/1661] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:47:54 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:47:55.394812 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e352f8-8754-4527-8dd4-5685cbfc37ef None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:55.414062 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e352f8-8754-4527-8dd4-5685cbfc37ef None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:55.421512 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00eaa7b-7e05-4469-9051-09bcb029a0f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:55.432668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00eaa7b-7e05-4469-9051-09bcb029a0f6 service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:55.433012 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00eaa7b-7e05-4469-9051-09bcb029a0f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:55.433349 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00eaa7b-7e05-4469-9051-09bcb029a0f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:55.601229 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:55.616215 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00eaa7b-7e05-4469-9051-09bcb029a0f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:55.619033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00eaa7b-7e05-4469-9051-09bcb029a0f6 service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:55.653343 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 828/1662] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:47:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:47:55.909141 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658cbd5-f941-42cf-963f-cfb7b5a19ac7 service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:55.910631 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-2658cbd5-f941-42cf-963f-cfb7b5a19ac7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:56.103201 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:56.116229 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658cbd5-f941-42cf-963f-cfb7b5a19ac7 service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:56.123376 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4e81b9-b238-4899-bfb6-3921d3726a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:56.125893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e81b9-b238-4899-bfb6-3921d3726a85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:56.129665 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e81b9-b238-4899-bfb6-3921d3726a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:56.129855 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e81b9-b238-4899-bfb6-3921d3726a85 admin admin] PATH_INFO: `/v3/projects/fcced7799400408083859f56c7b9181a` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:56.133345 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4e81b9-b238-4899-bfb6-3921d3726a85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcced7799400408083859f56c7b9181a)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:56.138251 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4e81b9-b238-4899-bfb6-3921d3726a85 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:56.244484 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fc4e81b9-b238-4899-bfb6-3921d3726a85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcced7799400408083859f56c7b9181a', 'request_id': 'req-fc4e81b9-b238-4899-bfb6-3921d3726a85'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:47:56.266564 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fc4e81b9-b238-4899-bfb6-3921d3726a85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcced7799400408083859f56c7b9181a', 'request_id': 'req-fc4e81b9-b238-4899-bfb6-3921d3726a85'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:47:56.296270 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fc4e81b9-b238-4899-bfb6-3921d3726a85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcced7799400408083859f56c7b9181a', 'request_id': 'req-fc4e81b9-b238-4899-bfb6-3921d3726a85'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:47:56.303281 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fc4e81b9-b238-4899-bfb6-3921d3726a85 admin admin] The token cache is being invalidate because project fcced7799400408083859f56c7b9181a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:47:56.303623 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fc4e81b9-b238-4899-bfb6-3921d3726a85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:47:56.304252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fc4e81b9-b238-4899-bfb6-3921d3726a85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:47:56.308991 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 835/1663] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:47:55 2022] DELETE /identity/v3/projects/fcced7799400408083859f56c7b9181a => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:47:56.310195 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00eaa7b-7e05-4469-9051-09bcb029a0f6 service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:56.325682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00eaa7b-7e05-4469-9051-09bcb029a0f6 service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:56.501908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5354279-58e2-406b-85a2-fe7c67384c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:56.504347 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5354279-58e2-406b-85a2-fe7c67384c51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:56.506847 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5354279-58e2-406b-85a2-fe7c67384c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:56.506847 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5354279-58e2-406b-85a2-fe7c67384c51 admin admin] PATH_INFO: `/v3/users/288d0774b20249ffb4c07535b0d8d349` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:56.511510 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5354279-58e2-406b-85a2-fe7c67384c51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=288d0774b20249ffb4c07535b0d8d349)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:56.516783 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5354279-58e2-406b-85a2-fe7c67384c51 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:56.619926 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-f5354279-58e2-406b-85a2-fe7c67384c51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '288d0774b20249ffb4c07535b0d8d349'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:47:56.649971 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-f5354279-58e2-406b-85a2-fe7c67384c51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '288d0774b20249ffb4c07535b0d8d349'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:47:56.650947 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-f5354279-58e2-406b-85a2-fe7c67384c51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '288d0774b20249ffb4c07535b0d8d349'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:47:56.653298 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-f5354279-58e2-406b-85a2-fe7c67384c51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '288d0774b20249ffb4c07535b0d8d349'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:47:56.668314 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-f5354279-58e2-406b-85a2-fe7c67384c51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '288d0774b20249ffb4c07535b0d8d349'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:47:56.714940 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 829/1664] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:47:56 2022] DELETE /identity/v3/users/288d0774b20249ffb4c07535b0d8d349 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:47:58.078381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4e81b9-b238-4899-bfb6-3921d3726a85 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:58.080965 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-fc4e81b9-b238-4899-bfb6-3921d3726a85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:58.117558 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4e81b9-b238-4899-bfb6-3921d3726a85 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:58.266500 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0a086-c2a8-4e35-b0e2-23219517116d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:58.268916 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0a086-c2a8-4e35-b0e2-23219517116d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:58.269407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0a086-c2a8-4e35-b0e2-23219517116d admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:58.270604 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0a086-c2a8-4e35-b0e2-23219517116d admin admin] PATH_INFO: `/v3/projects/3b92a2c7f66d470bbfb0db1da9b609bb` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:58.275706 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded0a086-c2a8-4e35-b0e2-23219517116d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b92a2c7f66d470bbfb0db1da9b609bb)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:58.284650 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded0a086-c2a8-4e35-b0e2-23219517116d admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:58.472955 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-ded0a086-c2a8-4e35-b0e2-23219517116d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b92a2c7f66d470bbfb0db1da9b609bb', 'request_id': 'req-ded0a086-c2a8-4e35-b0e2-23219517116d'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:47:58.494202 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-ded0a086-c2a8-4e35-b0e2-23219517116d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b92a2c7f66d470bbfb0db1da9b609bb', 'request_id': 'req-ded0a086-c2a8-4e35-b0e2-23219517116d'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:47:58.505808 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-ded0a086-c2a8-4e35-b0e2-23219517116d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b92a2c7f66d470bbfb0db1da9b609bb', 'request_id': 'req-ded0a086-c2a8-4e35-b0e2-23219517116d'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:47:58.526491 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-ded0a086-c2a8-4e35-b0e2-23219517116d admin admin] The token cache is being invalidate because project 3b92a2c7f66d470bbfb0db1da9b609bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:47:58.526835 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-ded0a086-c2a8-4e35-b0e2-23219517116d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:47:58.527721 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-ded0a086-c2a8-4e35-b0e2-23219517116d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:47:58.533730 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 836/1665] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:47:58 2022] DELETE /identity/v3/projects/3b92a2c7f66d470bbfb0db1da9b609bb => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:47:58.535242 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81cc3ad-f22d-45b4-8c68-269932746465 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:58.535760 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81cc3ad-f22d-45b4-8c68-269932746465 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:58.535988 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81cc3ad-f22d-45b4-8c68-269932746465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:58.573186 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-a81cc3ad-f22d-45b4-8c68-269932746465 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:58.576052 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-a81cc3ad-f22d-45b4-8c68-269932746465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:58.713872 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 830/1666] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:47:58.722843 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee4022-a16c-496e-9017-80e71ef50664 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:58.723164 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee4022-a16c-496e-9017-80e71ef50664 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:58.723446 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee4022-a16c-496e-9017-80e71ef50664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:58.766585 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-efee4022-a16c-496e-9017-80e71ef50664 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:58.768566 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-efee4022-a16c-496e-9017-80e71ef50664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:58.801373 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 837/1667] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:58.806273 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81cc3ad-f22d-45b4-8c68-269932746465 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:58.838652 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81cc3ad-f22d-45b4-8c68-269932746465 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:58.862796 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3310f57-ae89-4e36-8c98-e2e5a2384698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:58.863735 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3310f57-ae89-4e36-8c98-e2e5a2384698 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:58.864160 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3310f57-ae89-4e36-8c98-e2e5a2384698 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:58.864576 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3310f57-ae89-4e36-8c98-e2e5a2384698 admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:58.874388 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3310f57-ae89-4e36-8c98-e2e5a2384698 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:47:58.878653 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3310f57-ae89-4e36-8c98-e2e5a2384698 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:58.883150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3310f57-ae89-4e36-8c98-e2e5a2384698 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:58.911593 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:58.925368 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee4022-a16c-496e-9017-80e71ef50664 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:58.932676 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 831/1668] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:47:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:47:58.945662 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee4022-a16c-496e-9017-80e71ef50664 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:58.957785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f56b85-167d-4055-8026-036cae2b0215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:58.958715 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f56b85-167d-4055-8026-036cae2b0215 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:58.959216 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f56b85-167d-4055-8026-036cae2b0215 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:58.959644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f56b85-167d-4055-8026-036cae2b0215 admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:58.960652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f56b85-167d-4055-8026-036cae2b0215 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:58.964377 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f56b85-167d-4055-8026-036cae2b0215 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:59.001756 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:59.095193 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 838/1669] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:47:58 2022] POST /identity/v3/projects => generated 401 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:47:59.098393 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3310f57-ae89-4e36-8c98-e2e5a2384698 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:59.132210 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a68562-11d0-4ba8-97fd-3e6da8a9d530 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:59.133234 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a68562-11d0-4ba8-97fd-3e6da8a9d530 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:59.133544 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a68562-11d0-4ba8-97fd-3e6da8a9d530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:59.143578 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3310f57-ae89-4e36-8c98-e2e5a2384698 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:59.257708 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-03a68562-11d0-4ba8-97fd-3e6da8a9d530 None None] MFA Rules not processed for user `5373c0299ea64c9f872e3bc04bfb0b74`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:59.259869 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-03a68562-11d0-4ba8-97fd-3e6da8a9d530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:59.412676 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b1d92-15ae-448b-ac1f-29d11d878494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:59.413525 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b1d92-15ae-448b-ac1f-29d11d878494 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:59.414675 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b1d92-15ae-448b-ac1f-29d11d878494 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:59.415068 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b1d92-15ae-448b-ac1f-29d11d878494 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:59.415931 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6b1d92-15ae-448b-ac1f-29d11d878494 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:59.419008 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6b1d92-15ae-448b-ac1f-29d11d878494 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:59.450356 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 832/1670] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:47:59 2022] POST /identity/v3/users => generated 301 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:47:59.455553 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b1d92-15ae-448b-ac1f-29d11d878494 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:59.500463 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b1d92-15ae-448b-ac1f-29d11d878494 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:59.512541 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c89f4-b7ee-4bc4-ab9e-41ad823482e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:59.516936 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c89f4-b7ee-4bc4-ab9e-41ad823482e1 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:59.516936 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c89f4-b7ee-4bc4-ab9e-41ad823482e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:59.516936 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c89f4-b7ee-4bc4-ab9e-41ad823482e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:59.519406 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c89f4-b7ee-4bc4-ab9e-41ad823482e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:59.535071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c89f4-b7ee-4bc4-ab9e-41ad823482e1 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:59.539956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 839/1671] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:59.559215 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf660a-9946-400d-ac9c-9450d5ee840e None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:59.560236 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf660a-9946-400d-ac9c-9450d5ee840e None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:59.565449 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf660a-9946-400d-ac9c-9450d5ee840e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:59.601279 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 833/1672] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:47:59.602880 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c89f4-b7ee-4bc4-ab9e-41ad823482e1 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:59.644175 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-77bf660a-9946-400d-ac9c-9450d5ee840e None None] MFA Rules not processed for user `5373c0299ea64c9f872e3bc04bfb0b74`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:59.647079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-77bf660a-9946-400d-ac9c-9450d5ee840e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:59.652687 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c89f4-b7ee-4bc4-ab9e-41ad823482e1 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:59.708762 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f931526-9f53-4121-b2b1-24f5a7fbac66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:59.709576 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f931526-9f53-4121-b2b1-24f5a7fbac66 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:59.709658 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f931526-9f53-4121-b2b1-24f5a7fbac66 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:59.709908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f931526-9f53-4121-b2b1-24f5a7fbac66 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:59.710666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f931526-9f53-4121-b2b1-24f5a7fbac66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:59.712375 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f931526-9f53-4121-b2b1-24f5a7fbac66 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:59.714288 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 840/1673] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:47:59.727786 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf660a-9946-400d-ac9c-9450d5ee840e None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:59.729550 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-77bf660a-9946-400d-ac9c-9450d5ee840e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:47:59.735836 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 834/1674] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:47:59.740196 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f931526-9f53-4121-b2b1-24f5a7fbac66 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:47:59.757313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f931526-9f53-4121-b2b1-24f5a7fbac66 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:47:59.763957 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a26710-33fb-4eb6-9765-ffdbb8e79657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:47:59.765152 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a26710-33fb-4eb6-9765-ffdbb8e79657 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:59.765652 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a26710-33fb-4eb6-9765-ffdbb8e79657 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:59.766088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a26710-33fb-4eb6-9765-ffdbb8e79657 admin admin] PATH_INFO: `/v3/projects/e2716aaa6f1a4ea7a5440fe36c86e198/users/cdbf43bd21724f3f9120bbb4383e0717/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:59.808784 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a26710-33fb-4eb6-9765-ffdbb8e79657 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2716aaa6f1a4ea7a5440fe36c86e198, user_id=cdbf43bd21724f3f9120bbb4383e0717, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:47:59.813059 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a26710-33fb-4eb6-9765-ffdbb8e79657 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:47:59.860042 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 835/1675] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:47:59 2022] PUT /identity/v3/projects/e2716aaa6f1a4ea7a5440fe36c86e198/users/cdbf43bd21724f3f9120bbb4383e0717/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:47:59.870481 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628df54c-e2e9-4077-9421-42265fd5efdd None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:47:59.870754 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628df54c-e2e9-4077-9421-42265fd5efdd None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:47:59.871994 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628df54c-e2e9-4077-9421-42265fd5efdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:47:59.988786 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-628df54c-e2e9-4077-9421-42265fd5efdd None None] MFA Rules not processed for user `cdbf43bd21724f3f9120bbb4383e0717`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:47:59.990064 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-628df54c-e2e9-4077-9421-42265fd5efdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:00.058155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf660a-9946-400d-ac9c-9450d5ee840e None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:00.271004 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 836/1676] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:47:59 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:48:00.274010 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628df54c-e2e9-4077-9421-42265fd5efdd None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:00.277977 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-628df54c-e2e9-4077-9421-42265fd5efdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:00.297283 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21e035-46b6-44a3-aa06-207b10d7096a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:00.298096 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21e035-46b6-44a3-aa06-207b10d7096a service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:00.298213 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21e035-46b6-44a3-aa06-207b10d7096a service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:00.298843 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21e035-46b6-44a3-aa06-207b10d7096a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:00.314481 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:00.386858 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:00.482176 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628df54c-e2e9-4077-9421-42265fd5efdd None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:00.482417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e21e035-46b6-44a3-aa06-207b10d7096a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:00.484308 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e21e035-46b6-44a3-aa06-207b10d7096a service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:00.506935 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84074cc-b423-44cb-b585-78bdbd82ddde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:00.507895 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84074cc-b423-44cb-b585-78bdbd82ddde admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:00.508182 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84074cc-b423-44cb-b585-78bdbd82ddde admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:00.508510 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84074cc-b423-44cb-b585-78bdbd82ddde admin admin] PATH_INFO: `/v3/users/9f3b85f9d80446d9a4ad9674fc465879` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:00.510747 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84074cc-b423-44cb-b585-78bdbd82ddde admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f3b85f9d80446d9a4ad9674fc465879)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:00.518683 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84074cc-b423-44cb-b585-78bdbd82ddde admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:00.522963 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 841/1677] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:47:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 784 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:48:00.531941 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21e035-46b6-44a3-aa06-207b10d7096a service nova] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:00.568084 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21e035-46b6-44a3-aa06-207b10d7096a service nova] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:00.581061 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f9d4b-e1af-4b53-b007-56aa7b7ea7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:00.582055 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f9d4b-e1af-4b53-b007-56aa7b7ea7c7 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:00.582542 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f9d4b-e1af-4b53-b007-56aa7b7ea7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:00.582989 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f9d4b-e1af-4b53-b007-56aa7b7ea7c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:00.583904 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f9d4b-e1af-4b53-b007-56aa7b7ea7c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:00.588034 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f9d4b-e1af-4b53-b007-56aa7b7ea7c7 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:00.639114 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 842/1678] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:48:00 2022] POST /identity/v3/projects => generated 401 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:48:00.642480 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f9d4b-e1af-4b53-b007-56aa7b7ea7c7 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:00.664377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-b84074cc-b423-44cb-b585-78bdbd82ddde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f3b85f9d80446d9a4ad9674fc465879'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:00.711263 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-b84074cc-b423-44cb-b585-78bdbd82ddde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f3b85f9d80446d9a4ad9674fc465879'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:00.712058 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-b84074cc-b423-44cb-b585-78bdbd82ddde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f3b85f9d80446d9a4ad9674fc465879'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:00.715136 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-b84074cc-b423-44cb-b585-78bdbd82ddde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f3b85f9d80446d9a4ad9674fc465879'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:00.754464 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-b84074cc-b423-44cb-b585-78bdbd82ddde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f3b85f9d80446d9a4ad9674fc465879'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:00.802932 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 837/1679] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:48:00 2022] DELETE /identity/v3/users/9f3b85f9d80446d9a4ad9674fc465879 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:00.809535 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3ef4b-2ba4-441c-a3a5-1165f97f9bf7 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:00.809949 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3ef4b-2ba4-441c-a3a5-1165f97f9bf7 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:00.810273 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3ef4b-2ba4-441c-a3a5-1165f97f9bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:00.869103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f9d4b-e1af-4b53-b007-56aa7b7ea7c7 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:00.892634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-fcb3ef4b-2ba4-441c-a3a5-1165f97f9bf7 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:00.896260 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-fcb3ef4b-2ba4-441c-a3a5-1165f97f9bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:01.056777 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c880ecb-8460-4dc6-b6e9-6f8b7226f59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:01.057636 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c880ecb-8460-4dc6-b6e9-6f8b7226f59d admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:01.057930 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c880ecb-8460-4dc6-b6e9-6f8b7226f59d admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:01.058210 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c880ecb-8460-4dc6-b6e9-6f8b7226f59d admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:01.058931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c880ecb-8460-4dc6-b6e9-6f8b7226f59d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:01.060961 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c880ecb-8460-4dc6-b6e9-6f8b7226f59d admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:01.085650 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 838/1680] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:01.114601 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb3ef4b-2ba4-441c-a3a5-1165f97f9bf7 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:01.116198 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-fcb3ef4b-2ba4-441c-a3a5-1165f97f9bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:01.147359 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 843/1681] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:48:00 2022] POST /identity/v3/users => generated 307 bytes in 506 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:48:01.151887 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c880ecb-8460-4dc6-b6e9-6f8b7226f59d admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:01.160314 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-2c880ecb-8460-4dc6-b6e9-6f8b7226f59d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:01.186322 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c880ecb-8460-4dc6-b6e9-6f8b7226f59d admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:01.205058 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e38b5e-be96-4138-9df2-c1aab802b88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:01.205672 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e38b5e-be96-4138-9df2-c1aab802b88d admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:01.206323 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e38b5e-be96-4138-9df2-c1aab802b88d admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:01.206323 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e38b5e-be96-4138-9df2-c1aab802b88d admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:01.207002 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e38b5e-be96-4138-9df2-c1aab802b88d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:01.208850 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e38b5e-be96-4138-9df2-c1aab802b88d admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:01.235508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 844/1682] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:48:01.239303 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e38b5e-be96-4138-9df2-c1aab802b88d admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:01.250358 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb3ef4b-2ba4-441c-a3a5-1165f97f9bf7 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:01.261161 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923583f-a7a6-4bcb-ab80-b3a81ef8aacd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:01.263170 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923583f-a7a6-4bcb-ab80-b3a81ef8aacd service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:01.263774 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923583f-a7a6-4bcb-ab80-b3a81ef8aacd service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:01.265946 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923583f-a7a6-4bcb-ab80-b3a81ef8aacd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:01.269067 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e38b5e-be96-4138-9df2-c1aab802b88d admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:01.284726 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff46a0f7-79df-477f-a0a0-4097d1db8bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:01.286211 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46a0f7-79df-477f-a0a0-4097d1db8bb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:01.288104 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:01.288687 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46a0f7-79df-477f-a0a0-4097d1db8bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:01.289085 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46a0f7-79df-477f-a0a0-4097d1db8bb9 admin admin] PATH_INFO: `/v3/projects/887d1efa64f3408382a2a68b58e4c72d/users/44c2720bf38746b590304dc01d3bb3c8/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:01.293770 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c923583f-a7a6-4bcb-ab80-b3a81ef8aacd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:01.295653 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c923583f-a7a6-4bcb-ab80-b3a81ef8aacd service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:01.311456 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:01.329276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff46a0f7-79df-477f-a0a0-4097d1db8bb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=887d1efa64f3408382a2a68b58e4c72d, user_id=44c2720bf38746b590304dc01d3bb3c8, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:01.329646 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 839/1683] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:48:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:48:01.332278 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff46a0f7-79df-477f-a0a0-4097d1db8bb9 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:01.364156 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 845/1684] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:48:01 2022] PUT /identity/v3/projects/887d1efa64f3408382a2a68b58e4c72d/users/44c2720bf38746b590304dc01d3bb3c8/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:01.368412 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923583f-a7a6-4bcb-ab80-b3a81ef8aacd service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:01.385721 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923583f-a7a6-4bcb-ab80-b3a81ef8aacd service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:01.620928 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2089a92-508f-4c73-a347-2be6a88ccb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:01.621644 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2089a92-508f-4c73-a347-2be6a88ccb98 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:01.622090 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2089a92-508f-4c73-a347-2be6a88ccb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:01.622536 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2089a92-508f-4c73-a347-2be6a88ccb98 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:01.627735 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2089a92-508f-4c73-a347-2be6a88ccb98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:01.629156 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2089a92-508f-4c73-a347-2be6a88ccb98 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:01.658674 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 840/1685] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:48:01.661984 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff46a0f7-79df-477f-a0a0-4097d1db8bb9 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:01.684200 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff46a0f7-79df-477f-a0a0-4097d1db8bb9 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:01.699553 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc618c8-e17b-4655-9924-4e1c7c9593b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:01.700725 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc618c8-e17b-4655-9924-4e1c7c9593b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:01.701105 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc618c8-e17b-4655-9924-4e1c7c9593b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:01.701656 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc618c8-e17b-4655-9924-4e1c7c9593b7 admin admin] PATH_INFO: `/v3/projects/887d1efa64f3408382a2a68b58e4c72d/users/44c2720bf38746b590304dc01d3bb3c8/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:01.707612 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc618c8-e17b-4655-9924-4e1c7c9593b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=887d1efa64f3408382a2a68b58e4c72d, user_id=44c2720bf38746b590304dc01d3bb3c8, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:01.715827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc618c8-e17b-4655-9924-4e1c7c9593b7 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:01.762431 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 846/1686] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:48:01 2022] PUT /identity/v3/projects/887d1efa64f3408382a2a68b58e4c72d/users/44c2720bf38746b590304dc01d3bb3c8/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:01.773308 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44511fe-6c09-4318-87eb-63d92b7f93f1 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:01.773887 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44511fe-6c09-4318-87eb-63d92b7f93f1 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:01.773887 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44511fe-6c09-4318-87eb-63d92b7f93f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:01.838644 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-d44511fe-6c09-4318-87eb-63d92b7f93f1 None None] MFA Rules not processed for user `44c2720bf38746b590304dc01d3bb3c8`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:01.839943 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-d44511fe-6c09-4318-87eb-63d92b7f93f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:02.095871 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 841/1687] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:01 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:02.097210 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc618c8-e17b-4655-9924-4e1c7c9593b7 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:02.099904 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-fdc618c8-e17b-4655-9924-4e1c7c9593b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:02.252352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc618c8-e17b-4655-9924-4e1c7c9593b7 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:02.256755 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166ab51-c82d-49c0-bf88-eaffacf2b171 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:02.257922 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166ab51-c82d-49c0-bf88-eaffacf2b171 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:02.257922 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166ab51-c82d-49c0-bf88-eaffacf2b171 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:02.258340 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166ab51-c82d-49c0-bf88-eaffacf2b171 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:02.271245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0166ab51-c82d-49c0-bf88-eaffacf2b171 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:02.273395 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0166ab51-c82d-49c0-bf88-eaffacf2b171 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:02.287388 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 847/1688] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:48:02 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:48:02.293782 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a38ce-1acb-4354-96ce-069c90d8e834 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:02.293782 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a38ce-1acb-4354-96ce-069c90d8e834 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:02.294389 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a38ce-1acb-4354-96ce-069c90d8e834 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:02.296623 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 842/1689] 10.222.0.15 () {54 vars in 912 bytes} [Wed Jul 20 14:48:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:48:02.307457 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60eea1e-7d56-440a-aaef-ced5b1fdf251 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:02.308375 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60eea1e-7d56-440a-aaef-ced5b1fdf251 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:02.308852 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60eea1e-7d56-440a-aaef-ced5b1fdf251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:02.422837 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-e60eea1e-7d56-440a-aaef-ced5b1fdf251 None None] MFA Rules not processed for user `7658ffde740d446682c1876662815987`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:02.428389 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-e60eea1e-7d56-440a-aaef-ced5b1fdf251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:02.439154 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43a38ce-1acb-4354-96ce-069c90d8e834 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:02.473569 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43a38ce-1acb-4354-96ce-069c90d8e834 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:02.498725 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3de307-93ab-49fb-b882-14cddbcba5f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:02.499460 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3de307-93ab-49fb-b882-14cddbcba5f2 service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:02.499825 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3de307-93ab-49fb-b882-14cddbcba5f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:02.500110 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3de307-93ab-49fb-b882-14cddbcba5f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:02.529072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-8b3de307-93ab-49fb-b882-14cddbcba5f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:02.667707 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 848/1690] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Jul 20 14:48:02 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:02.669458 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8fceb-23a2-4503-8de0-68a1ea63056a None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:02.671899 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8fceb-23a2-4503-8de0-68a1ea63056a None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:02.672385 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8fceb-23a2-4503-8de0-68a1ea63056a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:02.719727 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3de307-93ab-49fb-b882-14cddbcba5f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:02.721260 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3de307-93ab-49fb-b882-14cddbcba5f2 service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:02.743468 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-84d8fceb-23a2-4503-8de0-68a1ea63056a None None] MFA Rules not processed for user `e3574b5b8f3241b192b0b810122d4769`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:02.747209 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-84d8fceb-23a2-4503-8de0-68a1ea63056a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:02.762789 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 843/1691] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:48:02 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:48:02.763990 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3de307-93ab-49fb-b882-14cddbcba5f2 service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:02.777220 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-8b3de307-93ab-49fb-b882-14cddbcba5f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:02.820230 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3de307-93ab-49fb-b882-14cddbcba5f2 service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:02.831824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09445cd8-3154-4b07-8235-6aab0fb9e0f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:02.832475 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09445cd8-3154-4b07-8235-6aab0fb9e0f8 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:02.832761 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09445cd8-3154-4b07-8235-6aab0fb9e0f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:02.833045 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09445cd8-3154-4b07-8235-6aab0fb9e0f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:02.858671 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09445cd8-3154-4b07-8235-6aab0fb9e0f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:02.861227 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09445cd8-3154-4b07-8235-6aab0fb9e0f8 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:02.911380 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 844/1692] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:48:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:48:02.968119 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 849/1693] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:02.982040 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09445cd8-3154-4b07-8235-6aab0fb9e0f8 service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:02.982788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-09445cd8-3154-4b07-8235-6aab0fb9e0f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:02.992023 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:03.163494 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8fceb-23a2-4503-8de0-68a1ea63056a None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:03.167849 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-84d8fceb-23a2-4503-8de0-68a1ea63056a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:03.184967 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:03.226205 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09445cd8-3154-4b07-8235-6aab0fb9e0f8 service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:03.249765 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce937e-937a-4e33-a470-d28257c8b907 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:03.250459 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce937e-937a-4e33-a470-d28257c8b907 service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:03.250743 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce937e-937a-4e33-a470-d28257c8b907 service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:03.267339 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce937e-937a-4e33-a470-d28257c8b907 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:03.299490 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ce937e-937a-4e33-a470-d28257c8b907 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:03.302493 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ce937e-937a-4e33-a470-d28257c8b907 service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:03.359999 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 845/1694] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:48:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:48:03.466538 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8fceb-23a2-4503-8de0-68a1ea63056a None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:03.507563 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea8e0d3-a416-4a32-b873-08ce8e794674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:03.512026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8e0d3-a416-4a32-b873-08ce8e794674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:03.522709 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8e0d3-a416-4a32-b873-08ce8e794674 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:03.523169 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8e0d3-a416-4a32-b873-08ce8e794674 admin admin] PATH_INFO: `/v3/projects/ad412de7ac43432593c0bddd2e1c426a` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:03.534127 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea8e0d3-a416-4a32-b873-08ce8e794674 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad412de7ac43432593c0bddd2e1c426a)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:03.543347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea8e0d3-a416-4a32-b873-08ce8e794674 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:03.675425 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1ea8e0d3-a416-4a32-b873-08ce8e794674 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad412de7ac43432593c0bddd2e1c426a', 'request_id': 'req-1ea8e0d3-a416-4a32-b873-08ce8e794674'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:03.681994 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1ea8e0d3-a416-4a32-b873-08ce8e794674 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad412de7ac43432593c0bddd2e1c426a', 'request_id': 'req-1ea8e0d3-a416-4a32-b873-08ce8e794674'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:03.699872 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1ea8e0d3-a416-4a32-b873-08ce8e794674 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad412de7ac43432593c0bddd2e1c426a', 'request_id': 'req-1ea8e0d3-a416-4a32-b873-08ce8e794674'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:03.733585 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1ea8e0d3-a416-4a32-b873-08ce8e794674 admin admin] The token cache is being invalidate because project ad412de7ac43432593c0bddd2e1c426a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:48:03.734093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1ea8e0d3-a416-4a32-b873-08ce8e794674 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:03.737210 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1ea8e0d3-a416-4a32-b873-08ce8e794674 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:03.743621 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 850/1695] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:48:03 2022] DELETE /identity/v3/projects/ad412de7ac43432593c0bddd2e1c426a => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:03.748829 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d478391-aef1-4819-b2d3-9b3bcea3cf1a None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:03.749132 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d478391-aef1-4819-b2d3-9b3bcea3cf1a None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:03.749417 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d478391-aef1-4819-b2d3-9b3bcea3cf1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:03.816751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-6d478391-aef1-4819-b2d3-9b3bcea3cf1a None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:03.829539 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302c2fb-5e1b-4b1a-839e-f3c254b60a7a None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:03.829668 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302c2fb-5e1b-4b1a-839e-f3c254b60a7a None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:03.829906 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302c2fb-5e1b-4b1a-839e-f3c254b60a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:03.832729 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-6d478391-aef1-4819-b2d3-9b3bcea3cf1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:03.906972 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-7302c2fb-5e1b-4b1a-839e-f3c254b60a7a None None] MFA Rules not processed for user `ee5a234904e64eb9b2faad85909d0507`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:03.913173 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-7302c2fb-5e1b-4b1a-839e-f3c254b60a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:04.052888 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 846/1696] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:04.058150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83943706-b1e3-49cc-96f7-fb496d061c5f None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:04.058597 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83943706-b1e3-49cc-96f7-fb496d061c5f None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:04.058959 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83943706-b1e3-49cc-96f7-fb496d061c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:04.121483 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-83943706-b1e3-49cc-96f7-fb496d061c5f None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:04.122721 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-83943706-b1e3-49cc-96f7-fb496d061c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:04.151065 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 847/1697] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:48:04.151601 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 851/1698] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:04.158299 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83943706-b1e3-49cc-96f7-fb496d061c5f None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:04.166840 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7302c2fb-5e1b-4b1a-839e-f3c254b60a7a None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:04.172724 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-7302c2fb-5e1b-4b1a-839e-f3c254b60a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:04.187760 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83943706-b1e3-49cc-96f7-fb496d061c5f None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:04.202074 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976f595-c5d1-4858-bbeb-8f03e200d06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:04.203218 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976f595-c5d1-4858-bbeb-8f03e200d06c admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:04.203704 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976f595-c5d1-4858-bbeb-8f03e200d06c admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:04.204175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976f595-c5d1-4858-bbeb-8f03e200d06c admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:04.205360 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976f595-c5d1-4858-bbeb-8f03e200d06c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:48:04.209507 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976f595-c5d1-4858-bbeb-8f03e200d06c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:04.211608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976f595-c5d1-4858-bbeb-8f03e200d06c admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:04.241005 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 848/1699] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:48:04.244960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:04.245520 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976f595-c5d1-4858-bbeb-8f03e200d06c admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:04.262851 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976f595-c5d1-4858-bbeb-8f03e200d06c admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:04.273373 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630647e-dfd8-4ab6-8102-891c3365e254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:04.274184 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630647e-dfd8-4ab6-8102-891c3365e254 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:04.274570 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630647e-dfd8-4ab6-8102-891c3365e254 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:04.274945 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630647e-dfd8-4ab6-8102-891c3365e254 admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:04.275803 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f630647e-dfd8-4ab6-8102-891c3365e254 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:04.277841 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f630647e-dfd8-4ab6-8102-891c3365e254 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:04.384248 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 849/1700] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:48:04 2022] POST /identity/v3/projects => generated 371 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 14:48:04.386027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630647e-dfd8-4ab6-8102-891c3365e254 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:04.396720 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7302c2fb-5e1b-4b1a-839e-f3c254b60a7a None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:04.563240 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:04.611400 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630647e-dfd8-4ab6-8102-891c3365e254 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:04.626438 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ac49c-0c49-4b06-9fbe-9dd95b32aaf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:04.632730 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ac49c-0c49-4b06-9fbe-9dd95b32aaf6 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:04.633132 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ac49c-0c49-4b06-9fbe-9dd95b32aaf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:04.633514 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ac49c-0c49-4b06-9fbe-9dd95b32aaf6 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:04.635573 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ac49c-0c49-4b06-9fbe-9dd95b32aaf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:04.639901 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e690c1-a9de-4d19-a6b9-2db410f9ea84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:04.646869 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ac49c-0c49-4b06-9fbe-9dd95b32aaf6 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:04.647271 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e690c1-a9de-4d19-a6b9-2db410f9ea84 service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:04.647772 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e690c1-a9de-4d19-a6b9-2db410f9ea84 service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:04.649576 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e690c1-a9de-4d19-a6b9-2db410f9ea84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:04.709427 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 850/1701] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:48:04 2022] POST /identity/v3/users => generated 286 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:48:04.711272 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce00472-1a1b-4659-8547-c0c247136e43 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:04.711569 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce00472-1a1b-4659-8547-c0c247136e43 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:04.711853 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce00472-1a1b-4659-8547-c0c247136e43 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:04.713297 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 851/1702] 10.222.0.15 () {54 vars in 961 bytes} [Wed Jul 20 14:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:48:04.714114 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce00472-1a1b-4659-8547-c0c247136e43 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:04.764128 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce00472-1a1b-4659-8547-c0c247136e43 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:04.768959 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a7e93-3dc5-43f7-b54c-08740734d5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:04.769648 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a7e93-3dc5-43f7-b54c-08740734d5ec admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:04.769932 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a7e93-3dc5-43f7-b54c-08740734d5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:04.770209 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a7e93-3dc5-43f7-b54c-08740734d5ec admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:04.770951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a7e93-3dc5-43f7-b54c-08740734d5ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:04.772632 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a7e93-3dc5-43f7-b54c-08740734d5ec admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:04.798008 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 852/1703] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:48:04.807763 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ee4eb-b629-4e54-8c9c-f9c5335c361a None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:04.808708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ee4eb-b629-4e54-8c9c-f9c5335c361a None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:04.808708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ee4eb-b629-4e54-8c9c-f9c5335c361a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:04.838009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e690c1-a9de-4d19-a6b9-2db410f9ea84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:04.839754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e690c1-a9de-4d19-a6b9-2db410f9ea84 service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:04.848352 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-f49ee4eb-b629-4e54-8c9c-f9c5335c361a None None] MFA Rules not processed for user `cb857f32bbc04d82bfa681bd5659765f`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:04.849659 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-f49ee4eb-b629-4e54-8c9c-f9c5335c361a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:04.865098 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 852/1704] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:48:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 699 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 14:48:04.866026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e690c1-a9de-4d19-a6b9-2db410f9ea84 service nova] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:04.896218 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e690c1-a9de-4d19-a6b9-2db410f9ea84 service nova] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:04.909791 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef6256-f081-4861-8676-eb856890fcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:04.910810 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef6256-f081-4861-8676-eb856890fcf9 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:04.910954 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef6256-f081-4861-8676-eb856890fcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:04.911269 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef6256-f081-4861-8676-eb856890fcf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:04.912049 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef6256-f081-4861-8676-eb856890fcf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:04.921092 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef6256-f081-4861-8676-eb856890fcf9 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:04.948738 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 853/1705] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:48:04.953438 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef6256-f081-4861-8676-eb856890fcf9 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:04.979291 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef6256-f081-4861-8676-eb856890fcf9 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:04.992258 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7bbe18-8135-43f7-812e-c37a42f5361d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:04.993246 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7bbe18-8135-43f7-812e-c37a42f5361d admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:04.993555 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7bbe18-8135-43f7-812e-c37a42f5361d admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:04.993853 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7bbe18-8135-43f7-812e-c37a42f5361d admin admin] PATH_INFO: `/v3/projects/37615a08a5764fa081a66d7abd465508/users/c4488cfcd1444ab7bcc8bce1e1a5597a/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:05.042687 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7bbe18-8135-43f7-812e-c37a42f5361d admin admin] RBAC: Authorizing `identity:create_grant(project_id=37615a08a5764fa081a66d7abd465508, user_id=c4488cfcd1444ab7bcc8bce1e1a5597a, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:05.046751 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7bbe18-8135-43f7-812e-c37a42f5361d admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:05.049096 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 853/1706] 10.222.0.15 () {58 vars in 1058 bytes} [Wed Jul 20 14:48:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:05.058414 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ee4eb-b629-4e54-8c9c-f9c5335c361a None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:05.070194 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 854/1707] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:48:04 2022] PUT /identity/v3/projects/37615a08a5764fa081a66d7abd465508/users/c4488cfcd1444ab7bcc8bce1e1a5597a/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 14:48:05.079958 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ee4eb-b629-4e54-8c9c-f9c5335c361a None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:05.082842 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27825b3-5c80-4670-9cd6-7b54a34003e5 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:05.083174 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27825b3-5c80-4670-9cd6-7b54a34003e5 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:05.083462 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27825b3-5c80-4670-9cd6-7b54a34003e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:05.131676 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-f27825b3-5c80-4670-9cd6-7b54a34003e5 None None] MFA Rules not processed for user `c4488cfcd1444ab7bcc8bce1e1a5597a`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:05.134409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-f27825b3-5c80-4670-9cd6-7b54a34003e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:05.150977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:05.229524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9fbb24-8e9c-444c-aa55-40e9625fbf07 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb857f32bbc04d82bfa681bd5659765f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:05.230387 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9fbb24-8e9c-444c-aa55-40e9625fbf07 service placement] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:05.230676 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9fbb24-8e9c-444c-aa55-40e9625fbf07 service placement] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:05.230976 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9fbb24-8e9c-444c-aa55-40e9625fbf07 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:05.236176 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-ba9fbb24-8e9c-444c-aa55-40e9625fbf07 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:05.255503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9fbb24-8e9c-444c-aa55-40e9625fbf07 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:05.257351 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 855/1708] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:05.257469 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9fbb24-8e9c-444c-aa55-40e9625fbf07 service placement] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:05.258475 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27825b3-5c80-4670-9cd6-7b54a34003e5 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:05.288822 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 854/1709] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:48:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 14:48:05.412647 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27825b3-5c80-4670-9cd6-7b54a34003e5 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:05.423258 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f028e9e-afc3-444e-ae1e-7412948b85b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:05.425764 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f028e9e-afc3-444e-ae1e-7412948b85b0 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:05.426186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f028e9e-afc3-444e-ae1e-7412948b85b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:05.427798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f028e9e-afc3-444e-ae1e-7412948b85b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:05.430158 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f028e9e-afc3-444e-ae1e-7412948b85b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:05.434038 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f028e9e-afc3-444e-ae1e-7412948b85b0 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:05.501530 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 856/1710] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:48:05 2022] POST /identity/v3/projects => generated 373 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:48:05.507624 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9fbb24-8e9c-444c-aa55-40e9625fbf07 service placement] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:05.525387 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9fbb24-8e9c-444c-aa55-40e9625fbf07 service placement] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:05.674084 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:05.693372 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e641f59-5767-4bfa-9016-5e20fbc9aaa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:05.694071 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e641f59-5767-4bfa-9016-5e20fbc9aaa2 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:05.694913 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e641f59-5767-4bfa-9016-5e20fbc9aaa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:05.694913 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e641f59-5767-4bfa-9016-5e20fbc9aaa2 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:05.695355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e641f59-5767-4bfa-9016-5e20fbc9aaa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:05.697313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e641f59-5767-4bfa-9016-5e20fbc9aaa2 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:05.730912 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 855/1711] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:48:05 2022] POST /identity/v3/users => generated 293 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:48:05.735619 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f028e9e-afc3-444e-ae1e-7412948b85b0 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:05.754546 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f028e9e-afc3-444e-ae1e-7412948b85b0 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:05.766374 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aaf4e2-c4e5-4623-b2c0-388192ae8238 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:05.767040 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aaf4e2-c4e5-4623-b2c0-388192ae8238 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:05.767040 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aaf4e2-c4e5-4623-b2c0-388192ae8238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:05.778391 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d354d6-722a-4bee-b3e5-529279cb1131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:05.779303 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d354d6-722a-4bee-b3e5-529279cb1131 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:05.780282 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d354d6-722a-4bee-b3e5-529279cb1131 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:05.781987 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d354d6-722a-4bee-b3e5-529279cb1131 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:05.784513 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d354d6-722a-4bee-b3e5-529279cb1131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:05.793930 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d354d6-722a-4bee-b3e5-529279cb1131 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:05.822933 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-18aaf4e2-c4e5-4623-b2c0-388192ae8238 None None] MFA Rules not processed for user `938083ff74d84224be142a26d8e17ad6`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:05.824187 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-18aaf4e2-c4e5-4623-b2c0-388192ae8238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:05.836781 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 857/1712] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:48:05.838732 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d354d6-722a-4bee-b3e5-529279cb1131 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:05.872723 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d354d6-722a-4bee-b3e5-529279cb1131 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:05.889137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea182d1-0b77-4243-a8cb-de7c3bf45233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:05.890445 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea182d1-0b77-4243-a8cb-de7c3bf45233 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:05.890833 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea182d1-0b77-4243-a8cb-de7c3bf45233 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:05.891275 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea182d1-0b77-4243-a8cb-de7c3bf45233 admin admin] PATH_INFO: `/v3/projects/565954bf521e4b8384c76d0d8ea04c36/users/7dacc888e41941a1907264e33cf049cc/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:05.934717 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea182d1-0b77-4243-a8cb-de7c3bf45233 admin admin] RBAC: Authorizing `identity:create_grant(project_id=565954bf521e4b8384c76d0d8ea04c36, user_id=7dacc888e41941a1907264e33cf049cc, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:05.943473 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea182d1-0b77-4243-a8cb-de7c3bf45233 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:05.974959 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 858/1713] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:48:05 2022] PUT /identity/v3/projects/565954bf521e4b8384c76d0d8ea04c36/users/7dacc888e41941a1907264e33cf049cc/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:05.979499 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea182d1-0b77-4243-a8cb-de7c3bf45233 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:06.079825 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 856/1714] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:48:06.081396 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf050668-1090-477f-a103-9ec4867966de None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:06.082063 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf050668-1090-477f-a103-9ec4867966de None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:06.082063 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf050668-1090-477f-a103-9ec4867966de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:06.119799 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea182d1-0b77-4243-a8cb-de7c3bf45233 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:06.126653 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed7806-7bf8-4272-a95c-465df8f99d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:06.127346 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-cf050668-1090-477f-a103-9ec4867966de None None] MFA Rules not processed for user `938083ff74d84224be142a26d8e17ad6`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:06.127984 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed7806-7bf8-4272-a95c-465df8f99d9b admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:06.128546 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed7806-7bf8-4272-a95c-465df8f99d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:06.128667 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-cf050668-1090-477f-a103-9ec4867966de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:06.128964 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed7806-7bf8-4272-a95c-465df8f99d9b admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:06.129998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed7806-7bf8-4272-a95c-465df8f99d9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:06.139278 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed7806-7bf8-4272-a95c-465df8f99d9b admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:06.149499 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:06.189475 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 859/1715] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:48:06.194468 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed7806-7bf8-4272-a95c-465df8f99d9b admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:06.226469 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed7806-7bf8-4272-a95c-465df8f99d9b admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:06.256025 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48312401-c9b2-4662-a405-6832061e82fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:06.259736 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48312401-c9b2-4662-a405-6832061e82fb admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:06.261257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48312401-c9b2-4662-a405-6832061e82fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:06.262661 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48312401-c9b2-4662-a405-6832061e82fb admin admin] PATH_INFO: `/v3/projects/565954bf521e4b8384c76d0d8ea04c36/users/7dacc888e41941a1907264e33cf049cc/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:06.286763 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48312401-c9b2-4662-a405-6832061e82fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=565954bf521e4b8384c76d0d8ea04c36, user_id=7dacc888e41941a1907264e33cf049cc, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:06.293357 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48312401-c9b2-4662-a405-6832061e82fb admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:06.317276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 860/1716] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:48:06 2022] PUT /identity/v3/projects/565954bf521e4b8384c76d0d8ea04c36/users/7dacc888e41941a1907264e33cf049cc/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:06.329930 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59f21d-2b5f-4d15-868d-4f9bd383aa87 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:06.330408 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59f21d-2b5f-4d15-868d-4f9bd383aa87 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:06.330775 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59f21d-2b5f-4d15-868d-4f9bd383aa87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:06.369001 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 857/1717] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:06.382694 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf050668-1090-477f-a103-9ec4867966de None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:06.414118 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-0c59f21d-2b5f-4d15-868d-4f9bd383aa87 None None] MFA Rules not processed for user `7dacc888e41941a1907264e33cf049cc`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:06.416003 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-0c59f21d-2b5f-4d15-868d-4f9bd383aa87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:06.589088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf050668-1090-477f-a103-9ec4867966de None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:06.620621 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:06.633790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 861/1718] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:06 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:06.646801 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59f21d-2b5f-4d15-868d-4f9bd383aa87 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:06.649963 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-0c59f21d-2b5f-4d15-868d-4f9bd383aa87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:06.794669 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a37934-51b5-41d4-a916-4581a98df603 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:06.795677 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a37934-51b5-41d4-a916-4581a98df603 service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:06.795966 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a37934-51b5-41d4-a916-4581a98df603 service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:06.796245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a37934-51b5-41d4-a916-4581a98df603 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:06.882030 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59f21d-2b5f-4d15-868d-4f9bd383aa87 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:06.888450 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3076dc9-e500-4584-9f05-168f02feb241 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:06.889138 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3076dc9-e500-4584-9f05-168f02feb241 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:06.889420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3076dc9-e500-4584-9f05-168f02feb241 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:06.889691 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3076dc9-e500-4584-9f05-168f02feb241 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:06.912162 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3076dc9-e500-4584-9f05-168f02feb241 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:06.916583 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3076dc9-e500-4584-9f05-168f02feb241 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:06.941502 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 862/1719] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:48:06 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:48:06.943325 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ed668-db42-4807-afab-b11515d0d225 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:06.943816 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ed668-db42-4807-afab-b11515d0d225 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:06.944173 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ed668-db42-4807-afab-b11515d0d225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:06.963821 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a37934-51b5-41d4-a916-4581a98df603 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:06.966719 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a37934-51b5-41d4-a916-4581a98df603 service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:06.996732 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 858/1720] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:48:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:48:07.028785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-409ed668-db42-4807-afab-b11515d0d225 None None] MFA Rules not processed for user `5f51124af1e846fb89552078c81afdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:07.030240 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-409ed668-db42-4807-afab-b11515d0d225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:07.221378 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 863/1721] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:07.874526 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a37934-51b5-41d4-a916-4581a98df603 service nova] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:07.876055 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-b4a37934-51b5-41d4-a916-4581a98df603 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:07.926072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a37934-51b5-41d4-a916-4581a98df603 service nova] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:07.975732 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669bf44-d181-44d0-9ce6-dcb310c66428 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:07.976702 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669bf44-d181-44d0-9ce6-dcb310c66428 service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:07.977140 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669bf44-d181-44d0-9ce6-dcb310c66428 service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:07.978449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669bf44-d181-44d0-9ce6-dcb310c66428 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:08.014938 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7669bf44-d181-44d0-9ce6-dcb310c66428 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:08.017006 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7669bf44-d181-44d0-9ce6-dcb310c66428 service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:08.048630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 859/1722] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:48:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:48:08.720854 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf8e44-2de9-47fe-9c62-2984f426d4a3 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:08.721399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf8e44-2de9-47fe-9c62-2984f426d4a3 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:08.721399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf8e44-2de9-47fe-9c62-2984f426d4a3 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:08.723990 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 864/1723] 10.222.0.15 () {54 vars in 912 bytes} [Wed Jul 20 14:48:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:48:08.744125 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef860087-9474-4a7f-ab25-31b0ea1c96cf None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:08.744778 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef860087-9474-4a7f-ab25-31b0ea1c96cf None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:08.745268 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef860087-9474-4a7f-ab25-31b0ea1c96cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:08.798020 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-ef860087-9474-4a7f-ab25-31b0ea1c96cf None None] MFA Rules not processed for user `7658ffde740d446682c1876662815987`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:08.798921 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-ef860087-9474-4a7f-ab25-31b0ea1c96cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:09.032987 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 860/1724] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Jul 20 14:48:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:48:09.124492 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf8e44-2de9-47fe-9c62-2984f426d4a3 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:09.126896 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-76bf8e44-2de9-47fe-9c62-2984f426d4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:09.146245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef860087-9474-4a7f-ab25-31b0ea1c96cf None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:09.161083 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:09.199326 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf8e44-2de9-47fe-9c62-2984f426d4a3 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:09.218326 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3af419-12ad-4e95-85b5-558e8c0d3528 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:09.220419 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3af419-12ad-4e95-85b5-558e8c0d3528 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:09.220953 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3af419-12ad-4e95-85b5-558e8c0d3528 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:09.223618 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3af419-12ad-4e95-85b5-558e8c0d3528 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:09.272092 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3af419-12ad-4e95-85b5-558e8c0d3528 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:09.274101 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3af419-12ad-4e95-85b5-558e8c0d3528 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:09.319460 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 865/1725] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:48:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:48:09.337309 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db93f2-1e19-4c8a-afb6-f5d4f58692d2 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:09.337863 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db93f2-1e19-4c8a-afb6-f5d4f58692d2 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:09.338282 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db93f2-1e19-4c8a-afb6-f5d4f58692d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:09.369204 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:09.416459 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef860087-9474-4a7f-ab25-31b0ea1c96cf None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:09.429615 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74c46b-f5b7-4bc0-b4c4-189b50b9ee17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:09.430540 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74c46b-f5b7-4bc0-b4c4-189b50b9ee17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:09.430826 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74c46b-f5b7-4bc0-b4c4-189b50b9ee17 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:09.431135 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74c46b-f5b7-4bc0-b4c4-189b50b9ee17 admin admin] PATH_INFO: `/v3/users/c4488cfcd1444ab7bcc8bce1e1a5597a` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:09.436740 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab74c46b-f5b7-4bc0-b4c4-189b50b9ee17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4488cfcd1444ab7bcc8bce1e1a5597a)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:09.441213 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab74c46b-f5b7-4bc0-b4c4-189b50b9ee17 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:09.509082 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-f0db93f2-1e19-4c8a-afb6-f5d4f58692d2 None None] MFA Rules not processed for user `2f8cf6d908314281a3ddc9d9513d39c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:09.511434 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-f0db93f2-1e19-4c8a-afb6-f5d4f58692d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:09.580764 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ab74c46b-f5b7-4bc0-b4c4-189b50b9ee17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4488cfcd1444ab7bcc8bce1e1a5597a'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:09.618237 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ab74c46b-f5b7-4bc0-b4c4-189b50b9ee17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4488cfcd1444ab7bcc8bce1e1a5597a'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:09.622073 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ab74c46b-f5b7-4bc0-b4c4-189b50b9ee17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4488cfcd1444ab7bcc8bce1e1a5597a'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:09.622925 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ab74c46b-f5b7-4bc0-b4c4-189b50b9ee17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4488cfcd1444ab7bcc8bce1e1a5597a'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:09.647470 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ab74c46b-f5b7-4bc0-b4c4-189b50b9ee17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4488cfcd1444ab7bcc8bce1e1a5597a'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:09.675887 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 866/1726] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Jul 20 14:48:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:48:09.686651 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 861/1727] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:48:09 2022] DELETE /identity/v3/users/c4488cfcd1444ab7bcc8bce1e1a5597a => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:09.694426 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0988c2-353a-4f79-85d1-8aeb011b7207 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:09.694977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0988c2-353a-4f79-85d1-8aeb011b7207 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:09.695555 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0988c2-353a-4f79-85d1-8aeb011b7207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:09.757044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-6b0988c2-353a-4f79-85d1-8aeb011b7207 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:09.759280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-6b0988c2-353a-4f79-85d1-8aeb011b7207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:09.846046 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 867/1728] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:09.989891 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74c46b-f5b7-4bc0-b4c4-189b50b9ee17 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:09.996283 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-ab74c46b-f5b7-4bc0-b4c4-189b50b9ee17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:10.007586 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0988c2-353a-4f79-85d1-8aeb011b7207 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:10.009559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-6b0988c2-353a-4f79-85d1-8aeb011b7207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:10.144414 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74c46b-f5b7-4bc0-b4c4-189b50b9ee17 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:10.146668 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:10.148395 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251dd0a-32ce-4223-89e7-2b1aa5746939 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:10.149074 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251dd0a-32ce-4223-89e7-2b1aa5746939 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:10.149561 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251dd0a-32ce-4223-89e7-2b1aa5746939 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:10.150055 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251dd0a-32ce-4223-89e7-2b1aa5746939 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:10.186069 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c251dd0a-32ce-4223-89e7-2b1aa5746939 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:10.190417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0988c2-353a-4f79-85d1-8aeb011b7207 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:10.200410 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f496f-c010-48fe-9088-d5b35c43eff0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f4844988474d7a91133d77906611a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:10.201246 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f496f-c010-48fe-9088-d5b35c43eff0 service swift] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:10.203041 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f496f-c010-48fe-9088-d5b35c43eff0 service swift] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:10.203471 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f496f-c010-48fe-9088-d5b35c43eff0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:10.206087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c251dd0a-32ce-4223-89e7-2b1aa5746939 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:10.239585 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 862/1729] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:48:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:48:10.337286 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f496f-c010-48fe-9088-d5b35c43eff0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:10.340740 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f496f-c010-48fe-9088-d5b35c43eff0 service swift] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:10.357143 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 868/1730] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jul 20 14:48:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 14:48:10.947718 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e2ab2-dead-4ac7-8e9e-af908a78e6bf None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:10.948016 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e2ab2-dead-4ac7-8e9e-af908a78e6bf None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:10.948306 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e2ab2-dead-4ac7-8e9e-af908a78e6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:10.987140 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-5f4e2ab2-dead-4ac7-8e9e-af908a78e6bf None None] MFA Rules not processed for user `e3574b5b8f3241b192b0b810122d4769`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:10.988472 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-5f4e2ab2-dead-4ac7-8e9e-af908a78e6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:11.194029 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 863/1731] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:11.590776 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d973758-608c-472c-963b-210f3577ddca None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:11.591929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d973758-608c-472c-963b-210f3577ddca None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:11.592237 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d973758-608c-472c-963b-210f3577ddca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:11.654772 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-3d973758-608c-472c-963b-210f3577ddca None None] MFA Rules not processed for user `ee5a234904e64eb9b2faad85909d0507`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:11.659554 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-3d973758-608c-472c-963b-210f3577ddca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:11.916256 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e2ab2-dead-4ac7-8e9e-af908a78e6bf None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:11.917965 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-5f4e2ab2-dead-4ac7-8e9e-af908a78e6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:11.941408 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e2ab2-dead-4ac7-8e9e-af908a78e6bf None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:11.970806 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df316054-5b3d-4df3-a0b0-727ec4bcce7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:11.972750 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df316054-5b3d-4df3-a0b0-727ec4bcce7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:11.973291 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df316054-5b3d-4df3-a0b0-727ec4bcce7b admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:11.973758 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df316054-5b3d-4df3-a0b0-727ec4bcce7b admin admin] PATH_INFO: `/v3/users/cdbf43bd21724f3f9120bbb4383e0717` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:11.976423 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df316054-5b3d-4df3-a0b0-727ec4bcce7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdbf43bd21724f3f9120bbb4383e0717)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:11.990697 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df316054-5b3d-4df3-a0b0-727ec4bcce7b admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:12.011062 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:12.092914 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 869/1732] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 499 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:12.115033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-df316054-5b3d-4df3-a0b0-727ec4bcce7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdbf43bd21724f3f9120bbb4383e0717'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:12.156845 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-df316054-5b3d-4df3-a0b0-727ec4bcce7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdbf43bd21724f3f9120bbb4383e0717'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:12.160950 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-df316054-5b3d-4df3-a0b0-727ec4bcce7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdbf43bd21724f3f9120bbb4383e0717'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:12.163575 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-df316054-5b3d-4df3-a0b0-727ec4bcce7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdbf43bd21724f3f9120bbb4383e0717'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:12.195514 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-df316054-5b3d-4df3-a0b0-727ec4bcce7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdbf43bd21724f3f9120bbb4383e0717'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:12.277443 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 864/1733] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:48:11 2022] DELETE /identity/v3/users/cdbf43bd21724f3f9120bbb4383e0717 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 14:48:12.283997 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1991fc-b177-4dd9-9e90-f583b5e2f637 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:12.284341 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1991fc-b177-4dd9-9e90-f583b5e2f637 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:12.284665 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1991fc-b177-4dd9-9e90-f583b5e2f637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:12.409786 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-3f1991fc-b177-4dd9-9e90-f583b5e2f637 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:12.423402 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-3f1991fc-b177-4dd9-9e90-f583b5e2f637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:12.764546 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 870/1734] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 483 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:12.891759 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df316054-5b3d-4df3-a0b0-727ec4bcce7b admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:12.896520 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-df316054-5b3d-4df3-a0b0-727ec4bcce7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:13.234221 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df316054-5b3d-4df3-a0b0-727ec4bcce7b admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:13.248873 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed742647-5535-477a-be3a-11770d056278 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:13.251190 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed742647-5535-477a-be3a-11770d056278 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:13.258257 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed742647-5535-477a-be3a-11770d056278 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:13.258665 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed742647-5535-477a-be3a-11770d056278 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:13.315156 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed742647-5535-477a-be3a-11770d056278 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:13.318780 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed742647-5535-477a-be3a-11770d056278 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:13.372803 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 865/1735] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:48:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:48:14.425285 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1991fc-b177-4dd9-9e90-f583b5e2f637 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:14.426958 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-3f1991fc-b177-4dd9-9e90-f583b5e2f637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:14.470762 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1991fc-b177-4dd9-9e90-f583b5e2f637 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:14.520349 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658a83b-a6a5-4e72-903a-f64fe9548072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:14.521752 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658a83b-a6a5-4e72-903a-f64fe9548072 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:14.523233 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658a83b-a6a5-4e72-903a-f64fe9548072 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:14.534310 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658a83b-a6a5-4e72-903a-f64fe9548072 admin admin] PATH_INFO: `/v3/projects/37615a08a5764fa081a66d7abd465508` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:14.539460 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3658a83b-a6a5-4e72-903a-f64fe9548072 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37615a08a5764fa081a66d7abd465508)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:14.547769 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3658a83b-a6a5-4e72-903a-f64fe9548072 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:14.573793 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ffab6-d1f6-4020-88c8-6402771851ff None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:14.574760 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ffab6-d1f6-4020-88c8-6402771851ff None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:14.576566 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ffab6-d1f6-4020-88c8-6402771851ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:14.646193 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-572ffab6-d1f6-4020-88c8-6402771851ff None None] MFA Rules not processed for user `7658ffde740d446682c1876662815987`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:14.651430 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-572ffab6-d1f6-4020-88c8-6402771851ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:14.679223 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 866/1736] 10.222.0.13 () {58 vars in 1015 bytes} [Wed Jul 20 14:48:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:14.687152 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:14.768742 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-3658a83b-a6a5-4e72-903a-f64fe9548072 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37615a08a5764fa081a66d7abd465508', 'request_id': 'req-3658a83b-a6a5-4e72-903a-f64fe9548072'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:14.809426 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-3658a83b-a6a5-4e72-903a-f64fe9548072 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37615a08a5764fa081a66d7abd465508', 'request_id': 'req-3658a83b-a6a5-4e72-903a-f64fe9548072'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:14.833312 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-3658a83b-a6a5-4e72-903a-f64fe9548072 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37615a08a5764fa081a66d7abd465508', 'request_id': 'req-3658a83b-a6a5-4e72-903a-f64fe9548072'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:14.857974 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ffab6-d1f6-4020-88c8-6402771851ff None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:14.877957 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ffab6-d1f6-4020-88c8-6402771851ff None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:14.892716 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-3658a83b-a6a5-4e72-903a-f64fe9548072 admin admin] The token cache is being invalidate because project 37615a08a5764fa081a66d7abd465508 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:48:14.893001 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-3658a83b-a6a5-4e72-903a-f64fe9548072 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:14.902081 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-3658a83b-a6a5-4e72-903a-f64fe9548072 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:14.911178 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 871/1737] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:48:14 2022] DELETE /identity/v3/projects/37615a08a5764fa081a66d7abd465508 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:14.915609 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658a83b-a6a5-4e72-903a-f64fe9548072 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:14.921799 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-3658a83b-a6a5-4e72-903a-f64fe9548072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:15.101415 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c1715-1e3a-47c2-990d-993cfd24c5a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:15.102914 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c1715-1e3a-47c2-990d-993cfd24c5a7 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:15.118045 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c1715-1e3a-47c2-990d-993cfd24c5a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:15.119140 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c1715-1e3a-47c2-990d-993cfd24c5a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:15.128756 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-a36c1715-1e3a-47c2-990d-993cfd24c5a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:15.159503 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658a83b-a6a5-4e72-903a-f64fe9548072 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:15.162270 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36c1715-1e3a-47c2-990d-993cfd24c5a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:15.164168 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36c1715-1e3a-47c2-990d-993cfd24c5a7 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:15.171713 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8880535-f870-407d-b291-6c6cecd5ff3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:15.184388 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8880535-f870-407d-b291-6c6cecd5ff3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:15.184816 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8880535-f870-407d-b291-6c6cecd5ff3d admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:15.185228 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8880535-f870-407d-b291-6c6cecd5ff3d admin admin] PATH_INFO: `/v3/users/7dacc888e41941a1907264e33cf049cc` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:15.187175 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8880535-f870-407d-b291-6c6cecd5ff3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dacc888e41941a1907264e33cf049cc)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:15.190408 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8880535-f870-407d-b291-6c6cecd5ff3d admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:15.191129 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 867/1738] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:48:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:48:15.323197 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-c8880535-f870-407d-b291-6c6cecd5ff3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dacc888e41941a1907264e33cf049cc'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:15.371667 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-c8880535-f870-407d-b291-6c6cecd5ff3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dacc888e41941a1907264e33cf049cc'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:15.375849 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-c8880535-f870-407d-b291-6c6cecd5ff3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dacc888e41941a1907264e33cf049cc'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:15.383744 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-c8880535-f870-407d-b291-6c6cecd5ff3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dacc888e41941a1907264e33cf049cc'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:15.404176 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-c8880535-f870-407d-b291-6c6cecd5ff3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dacc888e41941a1907264e33cf049cc'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:15.427185 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 872/1739] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:48:14 2022] DELETE /identity/v3/users/7dacc888e41941a1907264e33cf049cc => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:16.516994 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c1715-1e3a-47c2-990d-993cfd24c5a7 service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:16.519351 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-a36c1715-1e3a-47c2-990d-993cfd24c5a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:16.634351 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c1715-1e3a-47c2-990d-993cfd24c5a7 service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:16.811399 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:17.166727 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb932e13-fe46-4e22-a711-7dbabe8d4adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:17.171268 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb932e13-fe46-4e22-a711-7dbabe8d4adc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:17.171605 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb932e13-fe46-4e22-a711-7dbabe8d4adc admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:17.171899 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb932e13-fe46-4e22-a711-7dbabe8d4adc admin admin] PATH_INFO: `/v3/projects/e2716aaa6f1a4ea7a5440fe36c86e198` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:17.185859 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb932e13-fe46-4e22-a711-7dbabe8d4adc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2716aaa6f1a4ea7a5440fe36c86e198)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:17.198227 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb932e13-fe46-4e22-a711-7dbabe8d4adc admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:17.424199 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c0164-817f-4198-a3fc-20fa213085c3 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:17.425110 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c0164-817f-4198-a3fc-20fa213085c3 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:17.425290 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c0164-817f-4198-a3fc-20fa213085c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:17.461298 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-cb932e13-fe46-4e22-a711-7dbabe8d4adc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2716aaa6f1a4ea7a5440fe36c86e198', 'request_id': 'req-cb932e13-fe46-4e22-a711-7dbabe8d4adc'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:17.494617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-cb932e13-fe46-4e22-a711-7dbabe8d4adc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2716aaa6f1a4ea7a5440fe36c86e198', 'request_id': 'req-cb932e13-fe46-4e22-a711-7dbabe8d4adc'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:17.532398 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-cb932e13-fe46-4e22-a711-7dbabe8d4adc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2716aaa6f1a4ea7a5440fe36c86e198', 'request_id': 'req-cb932e13-fe46-4e22-a711-7dbabe8d4adc'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:17.566512 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-164c0164-817f-4198-a3fc-20fa213085c3 None None] MFA Rules not processed for user `938083ff74d84224be142a26d8e17ad6`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:17.586488 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-164c0164-817f-4198-a3fc-20fa213085c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:17.590050 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-cb932e13-fe46-4e22-a711-7dbabe8d4adc admin admin] The token cache is being invalidate because project e2716aaa6f1a4ea7a5440fe36c86e198 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:48:17.590356 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-cb932e13-fe46-4e22-a711-7dbabe8d4adc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:17.599611 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-cb932e13-fe46-4e22-a711-7dbabe8d4adc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:17.614792 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 868/1740] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:48:16 2022] DELETE /identity/v3/projects/e2716aaa6f1a4ea7a5440fe36c86e198 => generated 0 bytes in 1096 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:17.616875 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb932e13-fe46-4e22-a711-7dbabe8d4adc admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:17.627555 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-cb932e13-fe46-4e22-a711-7dbabe8d4adc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:17.734457 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:17.912070 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 873/1741] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 490 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:17.939908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb932e13-fe46-4e22-a711-7dbabe8d4adc admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:17.973275 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b5728-4167-4719-80ed-5204dc91751e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:17.974427 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b5728-4167-4719-80ed-5204dc91751e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:17.974852 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b5728-4167-4719-80ed-5204dc91751e admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:17.975282 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b5728-4167-4719-80ed-5204dc91751e admin admin] PATH_INFO: `/v3/users/44c2720bf38746b590304dc01d3bb3c8` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:17.981528 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5b5728-4167-4719-80ed-5204dc91751e admin admin] RBAC: Authorizing `identity:delete_user(user_id=44c2720bf38746b590304dc01d3bb3c8)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:17.984039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5b5728-4167-4719-80ed-5204dc91751e admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:18.165160 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-6a5b5728-4167-4719-80ed-5204dc91751e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44c2720bf38746b590304dc01d3bb3c8'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:18.202895 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:18.221970 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-6a5b5728-4167-4719-80ed-5204dc91751e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44c2720bf38746b590304dc01d3bb3c8'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:18.228328 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-6a5b5728-4167-4719-80ed-5204dc91751e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44c2720bf38746b590304dc01d3bb3c8'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:18.230316 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-6a5b5728-4167-4719-80ed-5204dc91751e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44c2720bf38746b590304dc01d3bb3c8'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:18.275072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-6a5b5728-4167-4719-80ed-5204dc91751e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44c2720bf38746b590304dc01d3bb3c8'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:18.303701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 869/1742] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:48:17 2022] DELETE /identity/v3/users/44c2720bf38746b590304dc01d3bb3c8 => generated 0 bytes in 687 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:18.520549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c0164-817f-4198-a3fc-20fa213085c3 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:18.522969 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-164c0164-817f-4198-a3fc-20fa213085c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:18.553508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c0164-817f-4198-a3fc-20fa213085c3 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:18.811942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6262898-e0c8-4c1b-ad72-30619b4f9a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:18.814656 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6262898-e0c8-4c1b-ad72-30619b4f9a91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:18.814956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6262898-e0c8-4c1b-ad72-30619b4f9a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:18.815314 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6262898-e0c8-4c1b-ad72-30619b4f9a91 admin admin] PATH_INFO: `/v3/projects/565954bf521e4b8384c76d0d8ea04c36` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:18.834988 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6262898-e0c8-4c1b-ad72-30619b4f9a91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=565954bf521e4b8384c76d0d8ea04c36)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:18.848363 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6262898-e0c8-4c1b-ad72-30619b4f9a91 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:19.042803 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a6262898-e0c8-4c1b-ad72-30619b4f9a91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '565954bf521e4b8384c76d0d8ea04c36', 'request_id': 'req-a6262898-e0c8-4c1b-ad72-30619b4f9a91'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:19.058849 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a6262898-e0c8-4c1b-ad72-30619b4f9a91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '565954bf521e4b8384c76d0d8ea04c36', 'request_id': 'req-a6262898-e0c8-4c1b-ad72-30619b4f9a91'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:19.105887 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a6262898-e0c8-4c1b-ad72-30619b4f9a91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '565954bf521e4b8384c76d0d8ea04c36', 'request_id': 'req-a6262898-e0c8-4c1b-ad72-30619b4f9a91'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:19.148968 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a6262898-e0c8-4c1b-ad72-30619b4f9a91 admin admin] The token cache is being invalidate because project 565954bf521e4b8384c76d0d8ea04c36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:48:19.149380 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a6262898-e0c8-4c1b-ad72-30619b4f9a91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:19.152345 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a6262898-e0c8-4c1b-ad72-30619b4f9a91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:19.155256 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 874/1743] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:48:18 2022] DELETE /identity/v3/projects/565954bf521e4b8384c76d0d8ea04c36 => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:19.162345 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6ab62-1b0a-40cd-ae8e-4b1ed6fa6156 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:19.162936 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6ab62-1b0a-40cd-ae8e-4b1ed6fa6156 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:19.162936 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6ab62-1b0a-40cd-ae8e-4b1ed6fa6156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:19.220321 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-bcf6ab62-1b0a-40cd-ae8e-4b1ed6fa6156 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:19.221640 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-bcf6ab62-1b0a-40cd-ae8e-4b1ed6fa6156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:19.484803 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 870/1744] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:19.493245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13894183-6a53-4659-96fb-ed22594e9b33 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:19.493989 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13894183-6a53-4659-96fb-ed22594e9b33 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:19.495352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13894183-6a53-4659-96fb-ed22594e9b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:19.529418 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-13894183-6a53-4659-96fb-ed22594e9b33 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:19.536339 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-13894183-6a53-4659-96fb-ed22594e9b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:19.557659 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 875/1745] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:19.562118 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf6ab62-1b0a-40cd-ae8e-4b1ed6fa6156 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:19.590266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf6ab62-1b0a-40cd-ae8e-4b1ed6fa6156 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:19.622872 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc193860-b812-4704-a441-bb02d7beb7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:19.623870 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc193860-b812-4704-a441-bb02d7beb7a8 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:19.624413 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc193860-b812-4704-a441-bb02d7beb7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:19.624809 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc193860-b812-4704-a441-bb02d7beb7a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:19.625681 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc193860-b812-4704-a441-bb02d7beb7a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:48:19.626114 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc193860-b812-4704-a441-bb02d7beb7a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:19.628174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc193860-b812-4704-a441-bb02d7beb7a8 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:19.662812 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 871/1746] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:48:19.665187 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13894183-6a53-4659-96fb-ed22594e9b33 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:19.686295 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13894183-6a53-4659-96fb-ed22594e9b33 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:19.780396 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8047fce-44d6-4c8a-9cb1-06e81e3038f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:19.781175 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8047fce-44d6-4c8a-9cb1-06e81e3038f3 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:19.790587 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8047fce-44d6-4c8a-9cb1-06e81e3038f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:19.790909 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8047fce-44d6-4c8a-9cb1-06e81e3038f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:19.805431 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8047fce-44d6-4c8a-9cb1-06e81e3038f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:19.825467 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8047fce-44d6-4c8a-9cb1-06e81e3038f3 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:19.893664 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a8cd8-5728-446d-b451-87145418c13b None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:19.894184 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a8cd8-5728-446d-b451-87145418c13b None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:19.894669 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a8cd8-5728-446d-b451-87145418c13b None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:19.904174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 872/1747] 10.222.0.15 () {54 vars in 921 bytes} [Wed Jul 20 14:48:19 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 14:48:19.932954 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe8080-f74a-4aff-8fe9-ed457754ff14 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:19.937791 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe8080-f74a-4aff-8fe9-ed457754ff14 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:19.938310 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe8080-f74a-4aff-8fe9-ed457754ff14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:19.987146 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-4fbe8080-f74a-4aff-8fe9-ed457754ff14 None None] MFA Rules not processed for user `5d4a107ed04348b59d20d1703dbe04eb`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:19.996901 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-4fbe8080-f74a-4aff-8fe9-ed457754ff14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:20.020888 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 876/1748] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:48:19 2022] POST /identity/v3/projects => generated 383 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:48:20.028617 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8047fce-44d6-4c8a-9cb1-06e81e3038f3 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:20.199740 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 873/1749] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Jul 20 14:48:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:48:20.228474 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe8080-f74a-4aff-8fe9-ed457754ff14 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:20.230097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-4fbe8080-f74a-4aff-8fe9-ed457754ff14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:20.278323 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe8080-f74a-4aff-8fe9-ed457754ff14 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:20.317937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:20.388072 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8047fce-44d6-4c8a-9cb1-06e81e3038f3 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:20.400211 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202ce32-eb6f-46e6-a56b-3b19f66b743a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:20.402227 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202ce32-eb6f-46e6-a56b-3b19f66b743a admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:20.403708 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202ce32-eb6f-46e6-a56b-3b19f66b743a admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:20.405064 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202ce32-eb6f-46e6-a56b-3b19f66b743a admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:20.406015 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3202ce32-eb6f-46e6-a56b-3b19f66b743a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:20.413785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3202ce32-eb6f-46e6-a56b-3b19f66b743a admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:20.463805 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:20.468715 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 877/1750] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:48:20 2022] POST /identity/v3/users => generated 292 bytes in 443 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:48:20.481998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202ce32-eb6f-46e6-a56b-3b19f66b743a admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:20.521455 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b8248-196e-4b74-9bd6-72cea17d1ca8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:20.524062 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b8248-196e-4b74-9bd6-72cea17d1ca8 service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:20.524062 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b8248-196e-4b74-9bd6-72cea17d1ca8 service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:20.524276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b8248-196e-4b74-9bd6-72cea17d1ca8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:20.533747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202ce32-eb6f-46e6-a56b-3b19f66b743a admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:20.557456 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c0da4-0ae8-445b-9ad8-c4841f115f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:20.560057 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c0da4-0ae8-445b-9ad8-c4841f115f1b admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:20.561262 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c0da4-0ae8-445b-9ad8-c4841f115f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:20.562558 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c0da4-0ae8-445b-9ad8-c4841f115f1b admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:20.563391 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246b8248-196e-4b74-9bd6-72cea17d1ca8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:20.563987 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847c0da4-0ae8-445b-9ad8-c4841f115f1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:20.566304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847c0da4-0ae8-445b-9ad8-c4841f115f1b admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:20.568567 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246b8248-196e-4b74-9bd6-72cea17d1ca8 service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:20.598414 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 878/1751] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:48:20.602922 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c0da4-0ae8-445b-9ad8-c4841f115f1b admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:20.606533 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 874/1752] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:48:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 14:48:20.627501 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c0da4-0ae8-445b-9ad8-c4841f115f1b admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:20.659970 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46994beb-c508-4dc8-8127-d836c3842623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:20.660605 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46994beb-c508-4dc8-8127-d836c3842623 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:20.661003 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46994beb-c508-4dc8-8127-d836c3842623 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:20.661397 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46994beb-c508-4dc8-8127-d836c3842623 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:20.679716 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46994beb-c508-4dc8-8127-d836c3842623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:20.682001 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46994beb-c508-4dc8-8127-d836c3842623 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:20.701581 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b8248-196e-4b74-9bd6-72cea17d1ca8 service nova] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:20.722920 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 879/1753] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:48:20.731242 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b8248-196e-4b74-9bd6-72cea17d1ca8 service nova] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:20.759681 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79063279-b6bc-4a8c-8185-d97f771eabd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:20.762197 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79063279-b6bc-4a8c-8185-d97f771eabd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:20.762832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79063279-b6bc-4a8c-8185-d97f771eabd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:20.764280 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79063279-b6bc-4a8c-8185-d97f771eabd4 admin admin] PATH_INFO: `/v3/projects/325b990e778243a0bbc35317ca1ab8b2/users/6bc9c2b8fb2b494bae7019e00c3e2a79/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:20.805303 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79063279-b6bc-4a8c-8185-d97f771eabd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=325b990e778243a0bbc35317ca1ab8b2, user_id=6bc9c2b8fb2b494bae7019e00c3e2a79, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:20.818245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79063279-b6bc-4a8c-8185-d97f771eabd4 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:20.854984 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 875/1754] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:48:20 2022] PUT /identity/v3/projects/325b990e778243a0bbc35317ca1ab8b2/users/6bc9c2b8fb2b494bae7019e00c3e2a79/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 14:48:20.869128 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5068d-d623-4af5-9bc5-574fb414a3b1 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:20.869504 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5068d-d623-4af5-9bc5-574fb414a3b1 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:20.869893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5068d-d623-4af5-9bc5-574fb414a3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:20.945733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-40e5068d-d623-4af5-9bc5-574fb414a3b1 None None] MFA Rules not processed for user `6bc9c2b8fb2b494bae7019e00c3e2a79`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:20.947189 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-40e5068d-d623-4af5-9bc5-574fb414a3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:21.110195 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79063279-b6bc-4a8c-8185-d97f771eabd4 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:21.112557 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 880/1755] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:21.253263 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79063279-b6bc-4a8c-8185-d97f771eabd4 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:21.258587 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d525fe2-75a4-4e3e-bce6-ce843cbb7fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:21.259467 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d525fe2-75a4-4e3e-bce6-ce843cbb7fbe admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:21.259866 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d525fe2-75a4-4e3e-bce6-ce843cbb7fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:21.260258 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d525fe2-75a4-4e3e-bce6-ce843cbb7fbe admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:21.261083 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d525fe2-75a4-4e3e-bce6-ce843cbb7fbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:21.263207 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d525fe2-75a4-4e3e-bce6-ce843cbb7fbe admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:21.361394 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 876/1756] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:48:21 2022] POST /identity/v3/projects => generated 383 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:48:21.363537 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e5068d-d623-4af5-9bc5-574fb414a3b1 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:21.380132 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e5068d-d623-4af5-9bc5-574fb414a3b1 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:21.628620 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6ac6c-5724-4975-853e-b5262b3ab666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:21.629504 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6ac6c-5724-4975-853e-b5262b3ab666 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:21.629920 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6ac6c-5724-4975-853e-b5262b3ab666 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:21.630380 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6ac6c-5724-4975-853e-b5262b3ab666 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:21.631164 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b6ac6c-5724-4975-853e-b5262b3ab666 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:21.633738 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b6ac6c-5724-4975-853e-b5262b3ab666 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:21.665082 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca32d1c-d4b1-4ac3-b801-d1a74255acc8 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:21.666077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca32d1c-d4b1-4ac3-b801-d1a74255acc8 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:21.666630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca32d1c-d4b1-4ac3-b801-d1a74255acc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:21.706121 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-4ca32d1c-d4b1-4ac3-b801-d1a74255acc8 None None] MFA Rules not processed for user `5f51124af1e846fb89552078c81afdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:21.706578 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 881/1757] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:48:21 2022] POST /identity/v3/users => generated 298 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:48:21.708647 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-4ca32d1c-d4b1-4ac3-b801-d1a74255acc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:21.709736 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6ac6c-5724-4975-853e-b5262b3ab666 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:21.763139 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6ac6c-5724-4975-853e-b5262b3ab666 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:21.789871 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e83e1a4-1939-42e5-a842-b92e9a486429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:21.790891 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e83e1a4-1939-42e5-a842-b92e9a486429 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:21.791349 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e83e1a4-1939-42e5-a842-b92e9a486429 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:21.793036 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e83e1a4-1939-42e5-a842-b92e9a486429 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:21.794084 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e83e1a4-1939-42e5-a842-b92e9a486429 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:21.805986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e83e1a4-1939-42e5-a842-b92e9a486429 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:21.855078 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 882/1758] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:48:21 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:48:21.856173 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e83e1a4-1939-42e5-a842-b92e9a486429 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:21.891037 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e83e1a4-1939-42e5-a842-b92e9a486429 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:21.917994 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a02841-4a49-44a2-9e3b-c3c11264ed85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:21.919121 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a02841-4a49-44a2-9e3b-c3c11264ed85 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:21.920948 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a02841-4a49-44a2-9e3b-c3c11264ed85 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:21.921288 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a02841-4a49-44a2-9e3b-c3c11264ed85 admin admin] PATH_INFO: `/v3/projects/1d6fe7b655f245dc8249804faffd0f2b/users/69b36626d99640f3b761a8181de7869d/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:21.942996 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 877/1759] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:48:21.958608 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:21.970001 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a02841-4a49-44a2-9e3b-c3c11264ed85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d6fe7b655f245dc8249804faffd0f2b, user_id=69b36626d99640f3b761a8181de7869d, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:21.972808 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a02841-4a49-44a2-9e3b-c3c11264ed85 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:21.999222 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 883/1760] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:48:21 2022] PUT /identity/v3/projects/1d6fe7b655f245dc8249804faffd0f2b/users/69b36626d99640f3b761a8181de7869d/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:22.003685 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca32d1c-d4b1-4ac3-b801-d1a74255acc8 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:22.040793 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca32d1c-d4b1-4ac3-b801-d1a74255acc8 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:22.164669 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:22.205363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ff2ab-bb37-4c09-bd44-040390561fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:22.206958 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ff2ab-bb37-4c09-bd44-040390561fdb admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:22.208154 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ff2ab-bb37-4c09-bd44-040390561fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:22.208458 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ff2ab-bb37-4c09-bd44-040390561fdb admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:22.211813 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ff2ab-bb37-4c09-bd44-040390561fdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:22.220334 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ff2ab-bb37-4c09-bd44-040390561fdb admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:22.261600 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 878/1761] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:48:22.265860 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a02841-4a49-44a2-9e3b-c3c11264ed85 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:22.287737 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a02841-4a49-44a2-9e3b-c3c11264ed85 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:22.306125 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8b773-69cf-4e5c-a199-2d9d21c703c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:22.307212 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8b773-69cf-4e5c-a199-2d9d21c703c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:22.307615 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8b773-69cf-4e5c-a199-2d9d21c703c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:22.308006 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8b773-69cf-4e5c-a199-2d9d21c703c8 admin admin] PATH_INFO: `/v3/projects/1d6fe7b655f245dc8249804faffd0f2b/users/69b36626d99640f3b761a8181de7869d/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:22.315937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a8b773-69cf-4e5c-a199-2d9d21c703c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d6fe7b655f245dc8249804faffd0f2b, user_id=69b36626d99640f3b761a8181de7869d, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:22.320011 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a8b773-69cf-4e5c-a199-2d9d21c703c8 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:22.365108 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 884/1762] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:48:22 2022] PUT /identity/v3/projects/1d6fe7b655f245dc8249804faffd0f2b/users/69b36626d99640f3b761a8181de7869d/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:22.368949 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f5add-1341-4433-a595-c056374b31c4 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:22.369347 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f5add-1341-4433-a595-c056374b31c4 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:22.369640 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f5add-1341-4433-a595-c056374b31c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:22.486868 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-5d2f5add-1341-4433-a595-c056374b31c4 None None] MFA Rules not processed for user `69b36626d99640f3b761a8181de7869d`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:22.489222 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-5d2f5add-1341-4433-a595-c056374b31c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:22.727547 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 879/1763] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:22 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:48:22.743822 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8b773-69cf-4e5c-a199-2d9d21c703c8 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:22.747760 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-02a8b773-69cf-4e5c-a199-2d9d21c703c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:22.876629 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8b773-69cf-4e5c-a199-2d9d21c703c8 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:22.880751 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002aca80-8a36-4c1d-b431-1a7e565bbdd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:22.881658 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002aca80-8a36-4c1d-b431-1a7e565bbdd0 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:22.882095 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002aca80-8a36-4c1d-b431-1a7e565bbdd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:22.882522 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002aca80-8a36-4c1d-b431-1a7e565bbdd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:22.913239 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002aca80-8a36-4c1d-b431-1a7e565bbdd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:22.914997 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002aca80-8a36-4c1d-b431-1a7e565bbdd0 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:22.951020 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 885/1764] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:48:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:48:23.027420 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f5add-1341-4433-a595-c056374b31c4 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:23.030295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-5d2f5add-1341-4433-a595-c056374b31c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:23.217160 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f5add-1341-4433-a595-c056374b31c4 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:23.237923 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e7b8b-1ec8-486e-80e8-8e610718c566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:23.238705 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e7b8b-1ec8-486e-80e8-8e610718c566 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:23.238705 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e7b8b-1ec8-486e-80e8-8e610718c566 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:23.239007 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e7b8b-1ec8-486e-80e8-8e610718c566 admin admin] PATH_INFO: `/v3/projects/887d1efa64f3408382a2a68b58e4c72d` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:23.252319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e7b8b-1ec8-486e-80e8-8e610718c566 admin admin] RBAC: Authorizing `identity:delete_project(project_id=887d1efa64f3408382a2a68b58e4c72d)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:23.255829 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e7b8b-1ec8-486e-80e8-8e610718c566 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:23.330054 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002aca80-8a36-4c1d-b431-1a7e565bbdd0 service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:23.340441 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002aca80-8a36-4c1d-b431-1a7e565bbdd0 service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:23.361387 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6bc347-6779-4635-a5e0-c603a4acb879 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:23.362066 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bc347-6779-4635-a5e0-c603a4acb879 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:23.362368 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bc347-6779-4635-a5e0-c603a4acb879 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:23.362624 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bc347-6779-4635-a5e0-c603a4acb879 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:23.507856 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-709e7b8b-1ec8-486e-80e8-8e610718c566 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '887d1efa64f3408382a2a68b58e4c72d', 'request_id': 'req-709e7b8b-1ec8-486e-80e8-8e610718c566'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:23.525766 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-709e7b8b-1ec8-486e-80e8-8e610718c566 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '887d1efa64f3408382a2a68b58e4c72d', 'request_id': 'req-709e7b8b-1ec8-486e-80e8-8e610718c566'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:23.569224 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-709e7b8b-1ec8-486e-80e8-8e610718c566 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '887d1efa64f3408382a2a68b58e4c72d', 'request_id': 'req-709e7b8b-1ec8-486e-80e8-8e610718c566'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:23.584195 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:23.585003 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6bc347-6779-4635-a5e0-c603a4acb879 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:23.586481 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6bc347-6779-4635-a5e0-c603a4acb879 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:23.634152 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-709e7b8b-1ec8-486e-80e8-8e610718c566 admin admin] The token cache is being invalidate because project 887d1efa64f3408382a2a68b58e4c72d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:48:23.635532 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-709e7b8b-1ec8-486e-80e8-8e610718c566 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:23.638848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-709e7b8b-1ec8-486e-80e8-8e610718c566 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:23.646045 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 880/1765] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:48:23 2022] DELETE /identity/v3/projects/887d1efa64f3408382a2a68b58e4c72d => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:23.653593 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504cbfcd-1161-4a6b-9198-ea7ceaa56197 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:23.653883 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504cbfcd-1161-4a6b-9198-ea7ceaa56197 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:23.654166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504cbfcd-1161-4a6b-9198-ea7ceaa56197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:23.656759 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 886/1766] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:48:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 14:48:23.716266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-504cbfcd-1161-4a6b-9198-ea7ceaa56197 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:23.717469 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-504cbfcd-1161-4a6b-9198-ea7ceaa56197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:23.942181 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 881/1767] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:23.953217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1980a-4d2d-46e1-977c-eb9a10609e27 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:23.954145 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1980a-4d2d-46e1-977c-eb9a10609e27 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:23.954623 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1980a-4d2d-46e1-977c-eb9a10609e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:23.994736 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-c2a1980a-4d2d-46e1-977c-eb9a10609e27 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:23.996977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-c2a1980a-4d2d-46e1-977c-eb9a10609e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:24.032601 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 887/1768] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:48:24.037184 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504cbfcd-1161-4a6b-9198-ea7ceaa56197 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:24.065945 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504cbfcd-1161-4a6b-9198-ea7ceaa56197 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:24.091030 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54872627-96a0-4a88-b4d1-ae37523a78e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:24.092273 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54872627-96a0-4a88-b4d1-ae37523a78e3 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:24.092730 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54872627-96a0-4a88-b4d1-ae37523a78e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:24.094655 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54872627-96a0-4a88-b4d1-ae37523a78e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:24.095643 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54872627-96a0-4a88-b4d1-ae37523a78e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:48:24.097160 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54872627-96a0-4a88-b4d1-ae37523a78e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:24.099496 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54872627-96a0-4a88-b4d1-ae37523a78e3 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:24.154143 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1980a-4d2d-46e1-977c-eb9a10609e27 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:24.167794 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1980a-4d2d-46e1-977c-eb9a10609e27 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:24.168796 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 882/1769] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:48:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:48:24.204510 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659db08-6113-40ba-8609-fedd80f00956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:24.205239 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659db08-6113-40ba-8609-fedd80f00956 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:24.205533 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659db08-6113-40ba-8609-fedd80f00956 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:24.206611 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659db08-6113-40ba-8609-fedd80f00956 admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:24.207703 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3659db08-6113-40ba-8609-fedd80f00956 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:24.215897 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3659db08-6113-40ba-8609-fedd80f00956 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:24.299212 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 888/1770] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:48:24 2022] POST /identity/v3/projects => generated 387 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:48:24.304331 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54872627-96a0-4a88-b4d1-ae37523a78e3 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:24.320903 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54872627-96a0-4a88-b4d1-ae37523a78e3 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:24.579841 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ec64e5-b72a-4678-8485-10096c7db6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:24.580750 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec64e5-b72a-4678-8485-10096c7db6a1 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:24.581046 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec64e5-b72a-4678-8485-10096c7db6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:24.581333 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec64e5-b72a-4678-8485-10096c7db6a1 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:24.584550 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ec64e5-b72a-4678-8485-10096c7db6a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:24.599457 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ec64e5-b72a-4678-8485-10096c7db6a1 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:24.670472 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 883/1771] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:48:24 2022] POST /identity/v3/users => generated 294 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:48:24.672992 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659db08-6113-40ba-8609-fedd80f00956 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:24.689923 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659db08-6113-40ba-8609-fedd80f00956 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:24.702368 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb08969-a092-4d51-b9e2-36140ca36076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:24.703141 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb08969-a092-4d51-b9e2-36140ca36076 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:24.703432 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb08969-a092-4d51-b9e2-36140ca36076 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:24.703714 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb08969-a092-4d51-b9e2-36140ca36076 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:24.704521 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb08969-a092-4d51-b9e2-36140ca36076 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:24.708404 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb08969-a092-4d51-b9e2-36140ca36076 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:24.734203 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 889/1772] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:48:24 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:48:24.739800 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ec64e5-b72a-4678-8485-10096c7db6a1 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:24.772702 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ec64e5-b72a-4678-8485-10096c7db6a1 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:24.802044 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838e1a4-e64e-4dd8-b29f-466edcd1b15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:24.803397 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838e1a4-e64e-4dd8-b29f-466edcd1b15b admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:24.803881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838e1a4-e64e-4dd8-b29f-466edcd1b15b admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:24.806426 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838e1a4-e64e-4dd8-b29f-466edcd1b15b admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:24.810905 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838e1a4-e64e-4dd8-b29f-466edcd1b15b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:24.817458 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838e1a4-e64e-4dd8-b29f-466edcd1b15b admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:24.847738 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 884/1773] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:48:24 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:48:24.848947 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb08969-a092-4d51-b9e2-36140ca36076 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:24.865845 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb08969-a092-4d51-b9e2-36140ca36076 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:24.908956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa0ec98-12c5-47db-8aa6-2c3141f40aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:24.910576 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0ec98-12c5-47db-8aa6-2c3141f40aef admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:24.911272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0ec98-12c5-47db-8aa6-2c3141f40aef admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:24.911782 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0ec98-12c5-47db-8aa6-2c3141f40aef admin admin] PATH_INFO: `/v3/projects/d2f110caf4714ea1863a03abcc8eb737/users/f1fa221e423f4a9c8e2916a913a68cc8/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:24.946774 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa0ec98-12c5-47db-8aa6-2c3141f40aef admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2f110caf4714ea1863a03abcc8eb737, user_id=f1fa221e423f4a9c8e2916a913a68cc8, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:24.949872 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa0ec98-12c5-47db-8aa6-2c3141f40aef admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:24.989782 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 890/1774] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:48:24 2022] PUT /identity/v3/projects/d2f110caf4714ea1863a03abcc8eb737/users/f1fa221e423f4a9c8e2916a913a68cc8/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 14:48:25.000579 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b4a5c-9226-4c22-b9ce-6cb49000d560 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:25.001029 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b4a5c-9226-4c22-b9ce-6cb49000d560 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:25.001481 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b4a5c-9226-4c22-b9ce-6cb49000d560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:25.092618 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-8d0b4a5c-9226-4c22-b9ce-6cb49000d560 None None] MFA Rules not processed for user `f1fa221e423f4a9c8e2916a913a68cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:25.093724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-8d0b4a5c-9226-4c22-b9ce-6cb49000d560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:25.295730 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:25.369943 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 885/1775] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:25.375737 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa0ec98-12c5-47db-8aa6-2c3141f40aef admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:25.480597 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:25.721867 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa0ec98-12c5-47db-8aa6-2c3141f40aef admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:25.740307 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03383c5-4400-460e-a636-a662133a7dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:25.741017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03383c5-4400-460e-a636-a662133a7dfb admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:25.741298 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03383c5-4400-460e-a636-a662133a7dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:25.741571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03383c5-4400-460e-a636-a662133a7dfb admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:25.742286 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03383c5-4400-460e-a636-a662133a7dfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:25.749798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03383c5-4400-460e-a636-a662133a7dfb admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:25.815857 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 891/1776] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:48:25 2022] POST /identity/v3/projects => generated 389 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:48:25.817217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b4a5c-9226-4c22-b9ce-6cb49000d560 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:25.832723 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b4a5c-9226-4c22-b9ce-6cb49000d560 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:26.009994 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bd440-1bbc-4c0d-bb1a-94f6be391896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:26.010772 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bd440-1bbc-4c0d-bb1a-94f6be391896 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:26.011109 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bd440-1bbc-4c0d-bb1a-94f6be391896 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:26.011449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bd440-1bbc-4c0d-bb1a-94f6be391896 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:26.012426 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876bd440-1bbc-4c0d-bb1a-94f6be391896 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:26.015586 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876bd440-1bbc-4c0d-bb1a-94f6be391896 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:26.072020 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 886/1777] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:48:25 2022] POST /identity/v3/users => generated 301 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:48:26.076563 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03383c5-4400-460e-a636-a662133a7dfb admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:26.093567 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03383c5-4400-460e-a636-a662133a7dfb admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:26.136618 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb5e35-4031-4d62-8c26-74175ecee08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:26.137354 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb5e35-4031-4d62-8c26-74175ecee08f admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:26.137639 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb5e35-4031-4d62-8c26-74175ecee08f admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:26.137920 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb5e35-4031-4d62-8c26-74175ecee08f admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:26.152626 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb5e35-4031-4d62-8c26-74175ecee08f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:26.154412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb5e35-4031-4d62-8c26-74175ecee08f admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:26.206333 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bd440-1bbc-4c0d-bb1a-94f6be391896 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:26.206519 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 892/1778] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:48:26.220350 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bd440-1bbc-4c0d-bb1a-94f6be391896 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:26.232997 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669cb182-06d3-4048-a3de-a6b9dc48aa2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:26.244339 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669cb182-06d3-4048-a3de-a6b9dc48aa2a admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:26.246202 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669cb182-06d3-4048-a3de-a6b9dc48aa2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:26.247799 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669cb182-06d3-4048-a3de-a6b9dc48aa2a admin admin] PATH_INFO: `/v3/projects/27bda55bae8e49798d161e10965d1043/users/9b3ce43499424f0e98dd79d1d362eec0/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:26.329460 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669cb182-06d3-4048-a3de-a6b9dc48aa2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=27bda55bae8e49798d161e10965d1043, user_id=9b3ce43499424f0e98dd79d1d362eec0, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:26.333484 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669cb182-06d3-4048-a3de-a6b9dc48aa2a admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:26.424924 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 887/1779] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:48:26 2022] PUT /identity/v3/projects/27bda55bae8e49798d161e10965d1043/users/9b3ce43499424f0e98dd79d1d362eec0/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 14:48:26.428816 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb5e35-4031-4d62-8c26-74175ecee08f admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:26.443612 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb5e35-4031-4d62-8c26-74175ecee08f admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:26.749783 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d8cfc-d65a-4fba-9908-c6b5738db037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:26.750563 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d8cfc-d65a-4fba-9908-c6b5738db037 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:26.750850 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d8cfc-d65a-4fba-9908-c6b5738db037 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:26.754541 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d8cfc-d65a-4fba-9908-c6b5738db037 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:26.757726 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35d8cfc-d65a-4fba-9908-c6b5738db037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:26.764971 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35d8cfc-d65a-4fba-9908-c6b5738db037 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:26.821737 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 893/1780] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:48:26.823241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669cb182-06d3-4048-a3de-a6b9dc48aa2a admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:26.883957 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669cb182-06d3-4048-a3de-a6b9dc48aa2a admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:26.951079 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a52aa-26f6-46ee-98dd-4416e50ab8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:26.971555 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a52aa-26f6-46ee-98dd-4416e50ab8f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:26.975955 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a52aa-26f6-46ee-98dd-4416e50ab8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:26.977884 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a52aa-26f6-46ee-98dd-4416e50ab8f5 admin admin] PATH_INFO: `/v3/projects/27bda55bae8e49798d161e10965d1043/users/9b3ce43499424f0e98dd79d1d362eec0/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:26.989961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a52aa-26f6-46ee-98dd-4416e50ab8f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27bda55bae8e49798d161e10965d1043, user_id=9b3ce43499424f0e98dd79d1d362eec0, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:27.010512 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a52aa-26f6-46ee-98dd-4416e50ab8f5 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:27.066089 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 888/1781] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:48:26 2022] PUT /identity/v3/projects/27bda55bae8e49798d161e10965d1043/users/9b3ce43499424f0e98dd79d1d362eec0/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:27.077192 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61553056-da68-4d2b-8086-2b10b8267090 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:27.077961 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61553056-da68-4d2b-8086-2b10b8267090 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:27.078100 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61553056-da68-4d2b-8086-2b10b8267090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:27.173343 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-61553056-da68-4d2b-8086-2b10b8267090 None None] MFA Rules not processed for user `9b3ce43499424f0e98dd79d1d362eec0`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:27.176632 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-61553056-da68-4d2b-8086-2b10b8267090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:27.436672 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 894/1782] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:27 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:27.451451 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a52aa-26f6-46ee-98dd-4416e50ab8f5 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:27.453858 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-f65a52aa-26f6-46ee-98dd-4416e50ab8f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:27.648632 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61553056-da68-4d2b-8086-2b10b8267090 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:27.650147 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-61553056-da68-4d2b-8086-2b10b8267090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:27.680746 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:27.721631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a52aa-26f6-46ee-98dd-4416e50ab8f5 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:27.728378 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea98f4-716a-49cb-9092-10388e0ccbb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:27.729326 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea98f4-716a-49cb-9092-10388e0ccbb7 service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:27.729820 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea98f4-716a-49cb-9092-10388e0ccbb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:27.732128 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea98f4-716a-49cb-9092-10388e0ccbb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:27.746280 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ea98f4-716a-49cb-9092-10388e0ccbb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:27.749161 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ea98f4-716a-49cb-9092-10388e0ccbb7 service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:27.764111 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 889/1783] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:48:27 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:48:27.845687 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61553056-da68-4d2b-8086-2b10b8267090 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:27.857841 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8220d-bf1a-4d00-8106-5688bfa75c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:27.858804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8220d-bf1a-4d00-8106-5688bfa75c83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:27.859311 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8220d-bf1a-4d00-8106-5688bfa75c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:27.859599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8220d-bf1a-4d00-8106-5688bfa75c83 admin admin] PATH_INFO: `/v3/users/6bc9c2b8fb2b494bae7019e00c3e2a79` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:27.860944 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa8220d-bf1a-4d00-8106-5688bfa75c83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bc9c2b8fb2b494bae7019e00c3e2a79)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:27.863157 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa8220d-bf1a-4d00-8106-5688bfa75c83 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:28.016030 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-9aa8220d-bf1a-4d00-8106-5688bfa75c83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bc9c2b8fb2b494bae7019e00c3e2a79'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:28.060319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea98f4-716a-49cb-9092-10388e0ccbb7 service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:28.071232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-9aa8220d-bf1a-4d00-8106-5688bfa75c83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bc9c2b8fb2b494bae7019e00c3e2a79'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:28.072643 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-9aa8220d-bf1a-4d00-8106-5688bfa75c83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bc9c2b8fb2b494bae7019e00c3e2a79'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:28.073671 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-9aa8220d-bf1a-4d00-8106-5688bfa75c83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bc9c2b8fb2b494bae7019e00c3e2a79'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:28.074439 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea98f4-716a-49cb-9092-10388e0ccbb7 service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:28.083479 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10270fc8-ea91-4713-8320-ffc401cc2f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:28.084862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10270fc8-ea91-4713-8320-ffc401cc2f40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:28.085608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10270fc8-ea91-4713-8320-ffc401cc2f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:28.086199 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10270fc8-ea91-4713-8320-ffc401cc2f40 admin admin] PATH_INFO: `/v3/users/f1fa221e423f4a9c8e2916a913a68cc8` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:28.089328 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10270fc8-ea91-4713-8320-ffc401cc2f40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1fa221e423f4a9c8e2916a913a68cc8)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:28.092182 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10270fc8-ea91-4713-8320-ffc401cc2f40 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:28.117700 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-9aa8220d-bf1a-4d00-8106-5688bfa75c83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bc9c2b8fb2b494bae7019e00c3e2a79'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:28.159916 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 895/1784] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:48:27 2022] DELETE /identity/v3/users/6bc9c2b8fb2b494bae7019e00c3e2a79 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:28.170612 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac573b56-8ccf-4a21-9650-74088b0c0408 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:28.171345 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac573b56-8ccf-4a21-9650-74088b0c0408 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:28.172108 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac573b56-8ccf-4a21-9650-74088b0c0408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:28.193787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:28.208993 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-10270fc8-ea91-4713-8320-ffc401cc2f40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1fa221e423f4a9c8e2916a913a68cc8'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:28.242952 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-10270fc8-ea91-4713-8320-ffc401cc2f40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1fa221e423f4a9c8e2916a913a68cc8'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:28.244096 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-10270fc8-ea91-4713-8320-ffc401cc2f40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1fa221e423f4a9c8e2916a913a68cc8'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:28.246370 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-10270fc8-ea91-4713-8320-ffc401cc2f40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1fa221e423f4a9c8e2916a913a68cc8'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:28.264224 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-10270fc8-ea91-4713-8320-ffc401cc2f40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1fa221e423f4a9c8e2916a913a68cc8'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:28.276884 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-ac573b56-8ccf-4a21-9650-74088b0c0408 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:28.280148 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-ac573b56-8ccf-4a21-9650-74088b0c0408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:28.288895 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 890/1785] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:48:28 2022] DELETE /identity/v3/users/f1fa221e423f4a9c8e2916a913a68cc8 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:28.305403 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570bf1c-f885-4345-ae5a-4e0f2b3a7383 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:28.305655 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570bf1c-f885-4345-ae5a-4e0f2b3a7383 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:28.306574 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570bf1c-f885-4345-ae5a-4e0f2b3a7383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:28.337808 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-7570bf1c-f885-4345-ae5a-4e0f2b3a7383 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:28.340545 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-7570bf1c-f885-4345-ae5a-4e0f2b3a7383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:28.455763 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 896/1786] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:28.477045 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac573b56-8ccf-4a21-9650-74088b0c0408 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:28.478625 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-ac573b56-8ccf-4a21-9650-74088b0c0408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:28.527649 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 891/1787] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:48:28.530873 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570bf1c-f885-4345-ae5a-4e0f2b3a7383 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:28.538049 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-7570bf1c-f885-4345-ae5a-4e0f2b3a7383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:28.566293 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:28.687416 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac573b56-8ccf-4a21-9650-74088b0c0408 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:28.694213 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570bf1c-f885-4345-ae5a-4e0f2b3a7383 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:28.716310 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2972a57-e211-4396-a1d0-363755dacd50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:28.717024 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2972a57-e211-4396-a1d0-363755dacd50 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:28.717265 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a73731-f0a5-43df-aed1-16fa1e32ef47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:28.717397 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2972a57-e211-4396-a1d0-363755dacd50 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:28.717645 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2972a57-e211-4396-a1d0-363755dacd50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:28.718365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a73731-f0a5-43df-aed1-16fa1e32ef47 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:28.718780 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a73731-f0a5-43df-aed1-16fa1e32ef47 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:28.719393 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a73731-f0a5-43df-aed1-16fa1e32ef47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:28.743627 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2972a57-e211-4396-a1d0-363755dacd50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:28.745167 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2972a57-e211-4396-a1d0-363755dacd50 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:28.763643 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a73731-f0a5-43df-aed1-16fa1e32ef47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:28.770044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a73731-f0a5-43df-aed1-16fa1e32ef47 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:28.775584 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 892/1788] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:48:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:48:28.797298 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 897/1789] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:48:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:48:29.406667 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2972a57-e211-4396-a1d0-363755dacd50 service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:29.409072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-f2972a57-e211-4396-a1d0-363755dacd50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:29.444469 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2972a57-e211-4396-a1d0-363755dacd50 service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:29.451189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df77422-6532-42dd-ae77-1b4740329e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:29.452623 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df77422-6532-42dd-ae77-1b4740329e31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:29.453283 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df77422-6532-42dd-ae77-1b4740329e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:29.453627 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df77422-6532-42dd-ae77-1b4740329e31 admin admin] PATH_INFO: `/v3/projects/d2f110caf4714ea1863a03abcc8eb737` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:29.455581 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df77422-6532-42dd-ae77-1b4740329e31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2f110caf4714ea1863a03abcc8eb737)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:29.457878 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df77422-6532-42dd-ae77-1b4740329e31 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:29.551848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-2df77422-6532-42dd-ae77-1b4740329e31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2f110caf4714ea1863a03abcc8eb737', 'request_id': 'req-2df77422-6532-42dd-ae77-1b4740329e31'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:29.558913 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:29.562348 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-2df77422-6532-42dd-ae77-1b4740329e31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2f110caf4714ea1863a03abcc8eb737', 'request_id': 'req-2df77422-6532-42dd-ae77-1b4740329e31'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:29.571475 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-2df77422-6532-42dd-ae77-1b4740329e31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2f110caf4714ea1863a03abcc8eb737', 'request_id': 'req-2df77422-6532-42dd-ae77-1b4740329e31'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:29.591263 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-2df77422-6532-42dd-ae77-1b4740329e31 admin admin] The token cache is being invalidate because project d2f110caf4714ea1863a03abcc8eb737 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:48:29.591936 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-2df77422-6532-42dd-ae77-1b4740329e31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:29.592999 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-2df77422-6532-42dd-ae77-1b4740329e31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:29.596344 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 893/1790] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:48:29 2022] DELETE /identity/v3/projects/d2f110caf4714ea1863a03abcc8eb737 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:29.600379 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a73731-f0a5-43df-aed1-16fa1e32ef47 service neutron] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:29.648927 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57530a6b-3a27-4982-a77c-e376086594bc None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:29.649218 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57530a6b-3a27-4982-a77c-e376086594bc None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:29.649487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57530a6b-3a27-4982-a77c-e376086594bc None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:29.652032 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 894/1791] 10.222.0.15 () {54 vars in 921 bytes} [Wed Jul 20 14:48:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 14:48:29.652540 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a73731-f0a5-43df-aed1-16fa1e32ef47 service neutron] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:29.665407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b324b3-cfde-4553-89a4-7de0f2e8f2f5 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:29.665781 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b324b3-cfde-4553-89a4-7de0f2e8f2f5 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:29.666295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b324b3-cfde-4553-89a4-7de0f2e8f2f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:29.748761 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-24b324b3-cfde-4553-89a4-7de0f2e8f2f5 None None] MFA Rules not processed for user `5d4a107ed04348b59d20d1703dbe04eb`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:29.756742 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-24b324b3-cfde-4553-89a4-7de0f2e8f2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:29.780483 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb01815-4f9c-4fe5-9609-af6a22da78e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:29.781528 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb01815-4f9c-4fe5-9609-af6a22da78e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:29.782235 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb01815-4f9c-4fe5-9609-af6a22da78e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:29.782617 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb01815-4f9c-4fe5-9609-af6a22da78e4 admin admin] PATH_INFO: `/v3/users/9b3ce43499424f0e98dd79d1d362eec0` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:29.784187 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb01815-4f9c-4fe5-9609-af6a22da78e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b3ce43499424f0e98dd79d1d362eec0)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:29.786504 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb01815-4f9c-4fe5-9609-af6a22da78e4 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:29.915338 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-2cb01815-4f9c-4fe5-9609-af6a22da78e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b3ce43499424f0e98dd79d1d362eec0'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:29.929894 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-2cb01815-4f9c-4fe5-9609-af6a22da78e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b3ce43499424f0e98dd79d1d362eec0'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:29.931036 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-2cb01815-4f9c-4fe5-9609-af6a22da78e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b3ce43499424f0e98dd79d1d362eec0'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:29.933377 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-2cb01815-4f9c-4fe5-9609-af6a22da78e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b3ce43499424f0e98dd79d1d362eec0'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:29.939233 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 895/1792] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Jul 20 14:48:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:29.940526 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b324b3-cfde-4553-89a4-7de0f2e8f2f5 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:29.944032 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-24b324b3-cfde-4553-89a4-7de0f2e8f2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:29.944691 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-2cb01815-4f9c-4fe5-9609-af6a22da78e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b3ce43499424f0e98dd79d1d362eec0'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:29.966937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 898/1793] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:48:29 2022] DELETE /identity/v3/users/9b3ce43499424f0e98dd79d1d362eec0 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:29.967947 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb01815-4f9c-4fe5-9609-af6a22da78e4 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:29.971072 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-2cb01815-4f9c-4fe5-9609-af6a22da78e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:29.980221 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b324b3-cfde-4553-89a4-7de0f2e8f2f5 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:30.099981 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f57da-1571-4a48-8b4f-8b2a34d4d97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:30.104280 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f57da-1571-4a48-8b4f-8b2a34d4d97b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:30.105879 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f57da-1571-4a48-8b4f-8b2a34d4d97b admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:30.106249 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f57da-1571-4a48-8b4f-8b2a34d4d97b admin admin] PATH_INFO: `/v3/projects/325b990e778243a0bbc35317ca1ab8b2` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:30.109653 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f57da-1571-4a48-8b4f-8b2a34d4d97b admin admin] RBAC: Authorizing `identity:delete_project(project_id=325b990e778243a0bbc35317ca1ab8b2)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:30.115020 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f57da-1571-4a48-8b4f-8b2a34d4d97b admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:30.125560 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:30.165586 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb01815-4f9c-4fe5-9609-af6a22da78e4 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:30.172778 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7fde2-c72b-46d0-916b-3113c12f2bc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:30.173638 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7fde2-c72b-46d0-916b-3113c12f2bc0 service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:30.173974 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7fde2-c72b-46d0-916b-3113c12f2bc0 service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:30.174269 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7fde2-c72b-46d0-916b-3113c12f2bc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:30.177544 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-31e7fde2-c72b-46d0-916b-3113c12f2bc0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:30.192941 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e7fde2-c72b-46d0-916b-3113c12f2bc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:30.194692 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e7fde2-c72b-46d0-916b-3113c12f2bc0 service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:30.208136 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 899/1794] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:48:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:48:30.222988 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-c74f57da-1571-4a48-8b4f-8b2a34d4d97b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '325b990e778243a0bbc35317ca1ab8b2', 'request_id': 'req-c74f57da-1571-4a48-8b4f-8b2a34d4d97b'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:30.245168 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-c74f57da-1571-4a48-8b4f-8b2a34d4d97b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '325b990e778243a0bbc35317ca1ab8b2', 'request_id': 'req-c74f57da-1571-4a48-8b4f-8b2a34d4d97b'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:30.262680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-c74f57da-1571-4a48-8b4f-8b2a34d4d97b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '325b990e778243a0bbc35317ca1ab8b2', 'request_id': 'req-c74f57da-1571-4a48-8b4f-8b2a34d4d97b'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:30.283524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-c74f57da-1571-4a48-8b4f-8b2a34d4d97b admin admin] The token cache is being invalidate because project 325b990e778243a0bbc35317ca1ab8b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:48:30.283883 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-c74f57da-1571-4a48-8b4f-8b2a34d4d97b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:30.285081 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-c74f57da-1571-4a48-8b4f-8b2a34d4d97b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:30.290728 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 896/1795] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:48:29 2022] DELETE /identity/v3/projects/325b990e778243a0bbc35317ca1ab8b2 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:30.292154 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7fde2-c72b-46d0-916b-3113c12f2bc0 service nova] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:30.294024 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-31e7fde2-c72b-46d0-916b-3113c12f2bc0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:30.313052 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7fde2-c72b-46d0-916b-3113c12f2bc0 service nova] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:30.394384 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025eac7-9089-4eab-9d2c-410956ef3198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:30.397371 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025eac7-9089-4eab-9d2c-410956ef3198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:30.397371 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025eac7-9089-4eab-9d2c-410956ef3198 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:30.397604 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025eac7-9089-4eab-9d2c-410956ef3198 admin admin] PATH_INFO: `/v3/users/69b36626d99640f3b761a8181de7869d` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:30.399355 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025eac7-9089-4eab-9d2c-410956ef3198 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69b36626d99640f3b761a8181de7869d)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:30.402411 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025eac7-9089-4eab-9d2c-410956ef3198 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:30.439232 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f57da-1571-4a48-8b4f-8b2a34d4d97b admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:30.441314 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-c74f57da-1571-4a48-8b4f-8b2a34d4d97b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:30.469786 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f57da-1571-4a48-8b4f-8b2a34d4d97b admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:30.503250 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca8be1-349e-4d1f-97a7-5e9b44974551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:30.504716 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca8be1-349e-4d1f-97a7-5e9b44974551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:30.505181 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca8be1-349e-4d1f-97a7-5e9b44974551 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:30.506530 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca8be1-349e-4d1f-97a7-5e9b44974551 admin admin] PATH_INFO: `/v3/projects/27bda55bae8e49798d161e10965d1043` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:30.508722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca8be1-349e-4d1f-97a7-5e9b44974551 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27bda55bae8e49798d161e10965d1043)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:30.512204 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a025eac7-9089-4eab-9d2c-410956ef3198 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69b36626d99640f3b761a8181de7869d'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:30.512777 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca8be1-349e-4d1f-97a7-5e9b44974551 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:30.517387 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:30.542085 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a025eac7-9089-4eab-9d2c-410956ef3198 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69b36626d99640f3b761a8181de7869d'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:30.547547 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a025eac7-9089-4eab-9d2c-410956ef3198 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69b36626d99640f3b761a8181de7869d'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:30.548271 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a025eac7-9089-4eab-9d2c-410956ef3198 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69b36626d99640f3b761a8181de7869d'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:30.557768 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a025eac7-9089-4eab-9d2c-410956ef3198 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69b36626d99640f3b761a8181de7869d'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:30.581095 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 900/1796] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:48:30 2022] DELETE /identity/v3/users/69b36626d99640f3b761a8181de7869d => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:30.587734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-3eca8be1-349e-4d1f-97a7-5e9b44974551 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27bda55bae8e49798d161e10965d1043', 'request_id': 'req-3eca8be1-349e-4d1f-97a7-5e9b44974551'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:30.593455 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-3eca8be1-349e-4d1f-97a7-5e9b44974551 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27bda55bae8e49798d161e10965d1043', 'request_id': 'req-3eca8be1-349e-4d1f-97a7-5e9b44974551'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:30.601161 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-3eca8be1-349e-4d1f-97a7-5e9b44974551 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27bda55bae8e49798d161e10965d1043', 'request_id': 'req-3eca8be1-349e-4d1f-97a7-5e9b44974551'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:30.616425 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-3eca8be1-349e-4d1f-97a7-5e9b44974551 admin admin] The token cache is being invalidate because project 27bda55bae8e49798d161e10965d1043 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:48:30.616723 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-3eca8be1-349e-4d1f-97a7-5e9b44974551 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:30.617484 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-3eca8be1-349e-4d1f-97a7-5e9b44974551 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:30.622413 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 897/1797] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:48:30 2022] DELETE /identity/v3/projects/27bda55bae8e49798d161e10965d1043 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:30.624384 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4eece-a30b-4a55-9a27-46ad140a3e22 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:30.624906 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4eece-a30b-4a55-9a27-46ad140a3e22 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:30.625133 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4eece-a30b-4a55-9a27-46ad140a3e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:30.646979 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-34e4eece-a30b-4a55-9a27-46ad140a3e22 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:30.648445 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-34e4eece-a30b-4a55-9a27-46ad140a3e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:30.710999 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:30.736112 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 901/1798] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:30.746130 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f0a36-fdf8-438f-8381-9718a1a8e704 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:30.747367 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f0a36-fdf8-438f-8381-9718a1a8e704 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:30.747367 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f0a36-fdf8-438f-8381-9718a1a8e704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:30.778094 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-542f0a36-fdf8-438f-8381-9718a1a8e704 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:30.780072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-542f0a36-fdf8-438f-8381-9718a1a8e704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:30.792715 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 898/1799] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:30.797558 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e4eece-a30b-4a55-9a27-46ad140a3e22 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:30.816411 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e4eece-a30b-4a55-9a27-46ad140a3e22 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:30.823673 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d8c77-7b3c-465e-9747-c2fb5fd5fd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:30.824795 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d8c77-7b3c-465e-9747-c2fb5fd5fd05 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:30.825322 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d8c77-7b3c-465e-9747-c2fb5fd5fd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:30.825822 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d8c77-7b3c-465e-9747-c2fb5fd5fd05 admin admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:30.827199 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d8c77-7b3c-465e-9747-c2fb5fd5fd05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:48:30.827681 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d8c77-7b3c-465e-9747-c2fb5fd5fd05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:30.830373 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d8c77-7b3c-465e-9747-c2fb5fd5fd05 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:30.850483 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 902/1800] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:48:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:48:30.855344 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f0a36-fdf8-438f-8381-9718a1a8e704 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:30.869403 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f0a36-fdf8-438f-8381-9718a1a8e704 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:30.876043 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138c72b-ce2e-4097-88bb-15f83bbf5d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:30.876783 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138c72b-ce2e-4097-88bb-15f83bbf5d75 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:30.877087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138c72b-ce2e-4097-88bb-15f83bbf5d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:30.877524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138c72b-ce2e-4097-88bb-15f83bbf5d75 admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:30.878318 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3138c72b-ce2e-4097-88bb-15f83bbf5d75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:30.880489 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3138c72b-ce2e-4097-88bb-15f83bbf5d75 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:30.927866 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d8c77-7b3c-465e-9747-c2fb5fd5fd05 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:30.933975 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 899/1801] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:48:30 2022] POST /identity/v3/projects => generated 409 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 14:48:30.940348 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d8c77-7b3c-465e-9747-c2fb5fd5fd05 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:30.968107 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138c72b-ce2e-4097-88bb-15f83bbf5d75 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:30.970178 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-3138c72b-ce2e-4097-88bb-15f83bbf5d75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:31.034207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b379e30-e850-4045-9884-9a27cc5bb944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:31.035877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b379e30-e850-4045-9884-9a27cc5bb944 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:31.035877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b379e30-e850-4045-9884-9a27cc5bb944 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:31.036359 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b379e30-e850-4045-9884-9a27cc5bb944 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:31.037512 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b379e30-e850-4045-9884-9a27cc5bb944 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:31.039944 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b379e30-e850-4045-9884-9a27cc5bb944 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:31.071092 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 903/1802] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:48:30 2022] POST /identity/v3/users => generated 305 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:48:31.075023 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b379e30-e850-4045-9884-9a27cc5bb944 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:31.088098 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b379e30-e850-4045-9884-9a27cc5bb944 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:31.088324 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138c72b-ce2e-4097-88bb-15f83bbf5d75 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:31.093345 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2366f0a-bef3-4b36-bdda-7804d37120a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:31.093700 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07c99c-c971-41a1-bd0b-09ce38007fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:31.094070 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2366f0a-bef3-4b36-bdda-7804d37120a0 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:31.094360 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2366f0a-bef3-4b36-bdda-7804d37120a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:31.094707 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07c99c-c971-41a1-bd0b-09ce38007fa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:31.094806 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2366f0a-bef3-4b36-bdda-7804d37120a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:31.094997 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07c99c-c971-41a1-bd0b-09ce38007fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:31.095335 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07c99c-c971-41a1-bd0b-09ce38007fa4 admin admin] PATH_INFO: `/v3/projects/1d6fe7b655f245dc8249804faffd0f2b` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:31.096110 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2366f0a-bef3-4b36-bdda-7804d37120a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:31.096664 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07c99c-c971-41a1-bd0b-09ce38007fa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d6fe7b655f245dc8249804faffd0f2b)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:31.097207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2366f0a-bef3-4b36-bdda-7804d37120a0 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:31.099424 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07c99c-c971-41a1-bd0b-09ce38007fa4 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:31.109409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 904/1803] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:48:31.113872 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2366f0a-bef3-4b36-bdda-7804d37120a0 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:31.125919 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2366f0a-bef3-4b36-bdda-7804d37120a0 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:31.132017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb49bb-7895-40fc-9e44-fb285a08be67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:31.133029 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb49bb-7895-40fc-9e44-fb285a08be67 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:31.133906 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb49bb-7895-40fc-9e44-fb285a08be67 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:31.134341 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb49bb-7895-40fc-9e44-fb285a08be67 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:31.135218 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eb49bb-7895-40fc-9e44-fb285a08be67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:31.137351 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eb49bb-7895-40fc-9e44-fb285a08be67 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:31.149544 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 905/1804] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:48:31.153992 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb49bb-7895-40fc-9e44-fb285a08be67 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:31.162465 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ea07c99c-c971-41a1-bd0b-09ce38007fa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d6fe7b655f245dc8249804faffd0f2b', 'request_id': 'req-ea07c99c-c971-41a1-bd0b-09ce38007fa4'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:31.164529 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb49bb-7895-40fc-9e44-fb285a08be67 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:31.169100 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ea07c99c-c971-41a1-bd0b-09ce38007fa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d6fe7b655f245dc8249804faffd0f2b', 'request_id': 'req-ea07c99c-c971-41a1-bd0b-09ce38007fa4'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:31.175809 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ea07c99c-c971-41a1-bd0b-09ce38007fa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d6fe7b655f245dc8249804faffd0f2b', 'request_id': 'req-ea07c99c-c971-41a1-bd0b-09ce38007fa4'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:31.177648 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:31.190865 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ea07c99c-c971-41a1-bd0b-09ce38007fa4 admin admin] The token cache is being invalidate because project 1d6fe7b655f245dc8249804faffd0f2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:48:31.191371 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ea07c99c-c971-41a1-bd0b-09ce38007fa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:31.192068 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ea07c99c-c971-41a1-bd0b-09ce38007fa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:31.193843 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 900/1805] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:48:30 2022] DELETE /identity/v3/projects/1d6fe7b655f245dc8249804faffd0f2b => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:31.199137 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d55d1-eb9d-439b-a9c6-6c6cf4aeb329 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:31.199624 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d55d1-eb9d-439b-a9c6-6c6cf4aeb329 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:31.199940 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d55d1-eb9d-439b-a9c6-6c6cf4aeb329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:31.219134 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-154d55d1-eb9d-439b-a9c6-6c6cf4aeb329 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:31.220348 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-154d55d1-eb9d-439b-a9c6-6c6cf4aeb329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:31.229731 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc31f8f-8fe1-4a7c-87ab-17dbbe29edd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:31.230730 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc31f8f-8fe1-4a7c-87ab-17dbbe29edd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:31.231007 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc31f8f-8fe1-4a7c-87ab-17dbbe29edd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:31.231323 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc31f8f-8fe1-4a7c-87ab-17dbbe29edd3 admin admin] PATH_INFO: `/v3/projects/78e39b55dfb4430f8ef1ccf553f6eae3/users/45a72ac2c45e423891dd78a20b84d52f/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:31.247020 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc31f8f-8fe1-4a7c-87ab-17dbbe29edd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78e39b55dfb4430f8ef1ccf553f6eae3, user_id=45a72ac2c45e423891dd78a20b84d52f, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:31.249763 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc31f8f-8fe1-4a7c-87ab-17dbbe29edd3 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:31.275378 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 906/1806] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:48:31 2022] PUT /identity/v3/projects/78e39b55dfb4430f8ef1ccf553f6eae3/users/45a72ac2c45e423891dd78a20b84d52f/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:31.277745 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4220f37-c0f8-439e-9d46-24da946ddbad None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:31.278125 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4220f37-c0f8-439e-9d46-24da946ddbad None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:31.278737 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4220f37-c0f8-439e-9d46-24da946ddbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:31.298815 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 901/1807] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:48:31.307343 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92054d07-e1f7-47c3-aca0-6b74cc6b1987 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:31.307988 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92054d07-e1f7-47c3-aca0-6b74cc6b1987 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:31.308435 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92054d07-e1f7-47c3-aca0-6b74cc6b1987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:31.318330 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-b4220f37-c0f8-439e-9d46-24da946ddbad None None] MFA Rules not processed for user `45a72ac2c45e423891dd78a20b84d52f`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:31.319707 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-b4220f37-c0f8-439e-9d46-24da946ddbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:31.328395 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-92054d07-e1f7-47c3-aca0-6b74cc6b1987 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:31.329648 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-92054d07-e1f7-47c3-aca0-6b74cc6b1987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:31.341297 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:31.388926 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 907/1808] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:31 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:31.394173 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4220f37-c0f8-439e-9d46-24da946ddbad None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:31.395119 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 902/1809] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:48:31.395669 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-b4220f37-c0f8-439e-9d46-24da946ddbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:31.399409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92054d07-e1f7-47c3-aca0-6b74cc6b1987 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:31.409001 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92054d07-e1f7-47c3-aca0-6b74cc6b1987 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:31.410899 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4220f37-c0f8-439e-9d46-24da946ddbad None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:31.413886 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd39a0-7880-443a-a1c1-f618986194db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:31.414553 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd39a0-7880-443a-a1c1-f618986194db admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:31.414858 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd39a0-7880-443a-a1c1-f618986194db admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:31.415216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd39a0-7880-443a-a1c1-f618986194db admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:31.415981 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afd39a0-7880-443a-a1c1-f618986194db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:48:31.416293 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afd39a0-7880-443a-a1c1-f618986194db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:31.416652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f13f7-a76f-4569-8120-5db4eaf01af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:31.417434 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f13f7-a76f-4569-8120-5db4eaf01af0 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:31.417789 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f13f7-a76f-4569-8120-5db4eaf01af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:31.417884 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afd39a0-7880-443a-a1c1-f618986194db admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:31.418203 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f13f7-a76f-4569-8120-5db4eaf01af0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:31.419028 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36f13f7-a76f-4569-8120-5db4eaf01af0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:31.422352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36f13f7-a76f-4569-8120-5db4eaf01af0 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:31.430413 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 903/1810] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:48:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:48:31.437156 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd39a0-7880-443a-a1c1-f618986194db admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:31.451358 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd39a0-7880-443a-a1c1-f618986194db admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:31.457607 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 908/1811] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:48:31 2022] POST /identity/v3/projects => generated 405 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 14:48:31.457768 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd57e5-af00-4dc6-9ee5-309137f41e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:31.457768 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd57e5-af00-4dc6-9ee5-309137f41e5b admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:31.458035 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd57e5-af00-4dc6-9ee5-309137f41e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:31.458291 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd57e5-af00-4dc6-9ee5-309137f41e5b admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:31.458851 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f13f7-a76f-4569-8120-5db4eaf01af0 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:31.459017 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd57e5-af00-4dc6-9ee5-309137f41e5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:31.461009 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd57e5-af00-4dc6-9ee5-309137f41e5b admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:31.509276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 904/1812] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:48:31 2022] POST /identity/v3/projects => generated 381 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 14:48:31.515719 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd57e5-af00-4dc6-9ee5-309137f41e5b admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:31.578905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f13f7-a76f-4569-8120-5db4eaf01af0 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:31.599639 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:31.605769 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd57e5-af00-4dc6-9ee5-309137f41e5b admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:31.614103 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a0ffa-a16f-4438-957a-6d806118b0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:31.614853 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a0ffa-a16f-4438-957a-6d806118b0d7 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:31.615170 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a0ffa-a16f-4438-957a-6d806118b0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:31.616243 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a0ffa-a16f-4438-957a-6d806118b0d7 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:31.627521 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a0ffa-a16f-4438-957a-6d806118b0d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:31.630843 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a0ffa-a16f-4438-957a-6d806118b0d7 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:31.673195 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 905/1813] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:48:31 2022] POST /identity/v3/users => generated 291 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 14:48:31.678096 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a0ffa-a16f-4438-957a-6d806118b0d7 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:31.693131 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a0ffa-a16f-4438-957a-6d806118b0d7 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:31.699751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb997a-5d3e-4796-ad35-45542c35e54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:31.700470 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb997a-5d3e-4796-ad35-45542c35e54e admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:31.700777 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb997a-5d3e-4796-ad35-45542c35e54e admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:31.701035 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb997a-5d3e-4796-ad35-45542c35e54e admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:31.701821 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb997a-5d3e-4796-ad35-45542c35e54e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:31.702303 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df6901-74b1-4a45-b6de-e6e01403aabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:31.703187 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df6901-74b1-4a45-b6de-e6e01403aabe admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:31.703745 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df6901-74b1-4a45-b6de-e6e01403aabe admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:31.703907 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb997a-5d3e-4796-ad35-45542c35e54e admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:31.704057 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df6901-74b1-4a45-b6de-e6e01403aabe admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:31.704787 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df6901-74b1-4a45-b6de-e6e01403aabe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:31.707307 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df6901-74b1-4a45-b6de-e6e01403aabe admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:31.729535 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 906/1814] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:48:31.735541 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb997a-5d3e-4796-ad35-45542c35e54e admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:31.754930 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 909/1815] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:48:31 2022] POST /identity/v3/users => generated 309 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:48:31.756485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df6901-74b1-4a45-b6de-e6e01403aabe admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:31.765040 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb997a-5d3e-4796-ad35-45542c35e54e admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:31.775891 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df6901-74b1-4a45-b6de-e6e01403aabe admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:31.776152 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a2436-d57d-401a-98b5-02009428b87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:31.777377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a2436-d57d-401a-98b5-02009428b87b admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:31.778117 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a2436-d57d-401a-98b5-02009428b87b admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:31.778821 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a2436-d57d-401a-98b5-02009428b87b admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:31.780730 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a2436-d57d-401a-98b5-02009428b87b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:31.784307 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a2436-d57d-401a-98b5-02009428b87b admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:31.784480 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ebef05-fc03-450d-9a82-ee1ca496437e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:31.785541 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ebef05-fc03-450d-9a82-ee1ca496437e admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:31.786009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ebef05-fc03-450d-9a82-ee1ca496437e admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:31.786514 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ebef05-fc03-450d-9a82-ee1ca496437e admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:31.787870 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ebef05-fc03-450d-9a82-ee1ca496437e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:31.791625 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ebef05-fc03-450d-9a82-ee1ca496437e admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:31.817951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 907/1816] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 14:48:31.818770 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 910/1817] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:48:31.833275 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a2436-d57d-401a-98b5-02009428b87b admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:31.833689 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ebef05-fc03-450d-9a82-ee1ca496437e admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:31.853972 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a2436-d57d-401a-98b5-02009428b87b admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:31.854690 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ebef05-fc03-450d-9a82-ee1ca496437e admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:31.860736 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d87530-a10b-460f-a64b-4de1cad60e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:31.861972 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d87530-a10b-460f-a64b-4de1cad60e0c admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:31.862361 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d87530-a10b-460f-a64b-4de1cad60e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:31.862732 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d87530-a10b-460f-a64b-4de1cad60e0c admin admin] PATH_INFO: `/v3/projects/afc05851b4824b03ab71604b1e75cbb9/users/5b99217fba44450791186bb8e6c7c54b/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:31.864120 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee940a6-f5b9-4d33-8be6-ba4d0d90bca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:31.865149 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee940a6-f5b9-4d33-8be6-ba4d0d90bca9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:31.865611 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee940a6-f5b9-4d33-8be6-ba4d0d90bca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:31.865830 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee940a6-f5b9-4d33-8be6-ba4d0d90bca9 admin admin] PATH_INFO: `/v3/projects/84d6eeddac2f47a7b84ed2e4c48cd8dd/users/2b6922bd46824c5dbf5dd2aa91215efc/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:31.884928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d87530-a10b-460f-a64b-4de1cad60e0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=afc05851b4824b03ab71604b1e75cbb9, user_id=5b99217fba44450791186bb8e6c7c54b, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:31.889504 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d87530-a10b-460f-a64b-4de1cad60e0c admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:31.905987 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee940a6-f5b9-4d33-8be6-ba4d0d90bca9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84d6eeddac2f47a7b84ed2e4c48cd8dd, user_id=2b6922bd46824c5dbf5dd2aa91215efc, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:31.910121 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee940a6-f5b9-4d33-8be6-ba4d0d90bca9 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:31.915091 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 911/1818] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:48:31 2022] PUT /identity/v3/projects/afc05851b4824b03ab71604b1e75cbb9/users/5b99217fba44450791186bb8e6c7c54b/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:31.918982 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d87530-a10b-460f-a64b-4de1cad60e0c admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:31.931575 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:31.933520 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 908/1819] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:48:31 2022] PUT /identity/v3/projects/84d6eeddac2f47a7b84ed2e4c48cd8dd/users/2b6922bd46824c5dbf5dd2aa91215efc/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:31.949440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e465d-c7a3-4a12-93a7-b74b1db17799 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:31.949440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e465d-c7a3-4a12-93a7-b74b1db17799 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:31.949647 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e465d-c7a3-4a12-93a7-b74b1db17799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:32.019423 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-9a6e465d-c7a3-4a12-93a7-b74b1db17799 None None] MFA Rules not processed for user `2b6922bd46824c5dbf5dd2aa91215efc`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:32.021560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-9a6e465d-c7a3-4a12-93a7-b74b1db17799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:32.086519 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d87530-a10b-460f-a64b-4de1cad60e0c admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:32.185817 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b374e4-8d0f-4abb-bf2a-c1b395c17944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:32.187380 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b374e4-8d0f-4abb-bf2a-c1b395c17944 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:32.187820 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b374e4-8d0f-4abb-bf2a-c1b395c17944 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:32.188410 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b374e4-8d0f-4abb-bf2a-c1b395c17944 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:32.189591 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b374e4-8d0f-4abb-bf2a-c1b395c17944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:32.191754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b374e4-8d0f-4abb-bf2a-c1b395c17944 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:32.192955 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 909/1820] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:32.193897 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e465d-c7a3-4a12-93a7-b74b1db17799 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:32.213857 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 912/1821] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 14:48:32.216054 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e465d-c7a3-4a12-93a7-b74b1db17799 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:32.219495 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b374e4-8d0f-4abb-bf2a-c1b395c17944 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:32.240496 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138a631-1cb6-4c3d-8325-53e6a03ee041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:32.242150 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b374e4-8d0f-4abb-bf2a-c1b395c17944 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:32.244106 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138a631-1cb6-4c3d-8325-53e6a03ee041 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:32.244611 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138a631-1cb6-4c3d-8325-53e6a03ee041 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:32.244996 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138a631-1cb6-4c3d-8325-53e6a03ee041 admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:32.245869 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f138a631-1cb6-4c3d-8325-53e6a03ee041 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:32.248375 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f138a631-1cb6-4c3d-8325-53e6a03ee041 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:32.250479 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a007b4cd-d29b-472a-bd3c-4ae56cd51603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:32.252407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007b4cd-d29b-472a-bd3c-4ae56cd51603 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:32.252742 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007b4cd-d29b-472a-bd3c-4ae56cd51603 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:32.253581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007b4cd-d29b-472a-bd3c-4ae56cd51603 admin admin] PATH_INFO: `/v3/projects/afc05851b4824b03ab71604b1e75cbb9/users/5b99217fba44450791186bb8e6c7c54b/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:32.258932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a007b4cd-d29b-472a-bd3c-4ae56cd51603 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afc05851b4824b03ab71604b1e75cbb9, user_id=5b99217fba44450791186bb8e6c7c54b, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:32.262801 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a007b4cd-d29b-472a-bd3c-4ae56cd51603 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:32.280705 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 913/1822] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:48:32 2022] PUT /identity/v3/projects/afc05851b4824b03ab71604b1e75cbb9/users/5b99217fba44450791186bb8e6c7c54b/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:32.298303 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feece8f-e6ad-4351-af83-1453894e8142 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:32.299621 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feece8f-e6ad-4351-af83-1453894e8142 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:32.300059 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feece8f-e6ad-4351-af83-1453894e8142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:32.305396 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 910/1823] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:48:32 2022] POST /identity/v3/projects => generated 379 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:48:32.312485 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138a631-1cb6-4c3d-8325-53e6a03ee041 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:32.346210 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-9feece8f-e6ad-4351-af83-1453894e8142 None None] MFA Rules not processed for user `5b99217fba44450791186bb8e6c7c54b`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:32.348502 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-9feece8f-e6ad-4351-af83-1453894e8142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:32.433798 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138a631-1cb6-4c3d-8325-53e6a03ee041 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:32.442427 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4743ca67-9fe8-4c0c-bb4e-16e1e38c24c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:32.443517 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4743ca67-9fe8-4c0c-bb4e-16e1e38c24c2 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:32.443942 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4743ca67-9fe8-4c0c-bb4e-16e1e38c24c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:32.444358 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4743ca67-9fe8-4c0c-bb4e-16e1e38c24c2 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:32.445249 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4743ca67-9fe8-4c0c-bb4e-16e1e38c24c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:32.449523 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4743ca67-9fe8-4c0c-bb4e-16e1e38c24c2 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:32.463868 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 914/1824] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:32 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:48:32.471678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feece8f-e6ad-4351-af83-1453894e8142 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:32.473998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-9feece8f-e6ad-4351-af83-1453894e8142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:32.485705 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 911/1825] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:48:32 2022] POST /identity/v3/users => generated 296 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:48:32.489969 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4743ca67-9fe8-4c0c-bb4e-16e1e38c24c2 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:32.491088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:32.504661 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4743ca67-9fe8-4c0c-bb4e-16e1e38c24c2 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:32.511304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:32.513094 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483586f6-037f-43ba-afae-2015857ad7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:32.516768 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483586f6-037f-43ba-afae-2015857ad7ce admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:32.521007 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483586f6-037f-43ba-afae-2015857ad7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:32.521617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483586f6-037f-43ba-afae-2015857ad7ce admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:32.522898 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483586f6-037f-43ba-afae-2015857ad7ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:32.526181 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483586f6-037f-43ba-afae-2015857ad7ce admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:32.541308 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 912/1826] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 14:48:32.548351 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483586f6-037f-43ba-afae-2015857ad7ce admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:32.565052 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483586f6-037f-43ba-afae-2015857ad7ce admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:32.571959 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5970d7-7186-45e9-8342-f15f5c366fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:32.573114 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5970d7-7186-45e9-8342-f15f5c366fd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:32.574007 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5970d7-7186-45e9-8342-f15f5c366fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:32.574419 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5970d7-7186-45e9-8342-f15f5c366fd7 admin admin] PATH_INFO: `/v3/projects/d598fdd196454ba2a4defaaa7e674f0e/users/79c0cf5642504370b3f8897ba4d4cfb6/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:32.595621 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5970d7-7186-45e9-8342-f15f5c366fd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d598fdd196454ba2a4defaaa7e674f0e, user_id=79c0cf5642504370b3f8897ba4d4cfb6, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:32.597682 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feece8f-e6ad-4351-af83-1453894e8142 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:32.600624 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5970d7-7186-45e9-8342-f15f5c366fd7 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:32.604071 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6450b02e-6ee5-490f-97ab-6c4b2369955e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:32.606284 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450b02e-6ee5-490f-97ab-6c4b2369955e service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:32.606700 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450b02e-6ee5-490f-97ab-6c4b2369955e service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:32.606991 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450b02e-6ee5-490f-97ab-6c4b2369955e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:32.627799 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 913/1827] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:48:32 2022] PUT /identity/v3/projects/d598fdd196454ba2a4defaaa7e674f0e/users/79c0cf5642504370b3f8897ba4d4cfb6/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 14:48:32.629101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5970d7-7186-45e9-8342-f15f5c366fd7 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:32.630669 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6450b02e-6ee5-490f-97ab-6c4b2369955e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:32.632310 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6450b02e-6ee5-490f-97ab-6c4b2369955e service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:32.656430 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 915/1828] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:48:32 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:48:32.746525 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5970d7-7186-45e9-8342-f15f5c366fd7 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:32.752038 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d004909-ca45-4338-8631-1c4179e60f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:32.754091 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d004909-ca45-4338-8631-1c4179e60f57 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:32.754384 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d004909-ca45-4338-8631-1c4179e60f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:32.754668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d004909-ca45-4338-8631-1c4179e60f57 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:32.755455 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d004909-ca45-4338-8631-1c4179e60f57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:32.757188 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d004909-ca45-4338-8631-1c4179e60f57 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:32.772096 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 914/1829] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:48:32.773015 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6450b02e-6ee5-490f-97ab-6c4b2369955e service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:32.788296 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6450b02e-6ee5-490f-97ab-6c4b2369955e service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:32.803558 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae37f13-ba76-4799-8f41-16d435c6507b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:32.806331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae37f13-ba76-4799-8f41-16d435c6507b admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:32.806614 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae37f13-ba76-4799-8f41-16d435c6507b admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:32.806939 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae37f13-ba76-4799-8f41-16d435c6507b admin admin] PATH_INFO: `/v3/projects/d598fdd196454ba2a4defaaa7e674f0e/users/79c0cf5642504370b3f8897ba4d4cfb6/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:32.814810 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae37f13-ba76-4799-8f41-16d435c6507b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d598fdd196454ba2a4defaaa7e674f0e, user_id=79c0cf5642504370b3f8897ba4d4cfb6, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:32.817839 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae37f13-ba76-4799-8f41-16d435c6507b admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:32.850355 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 916/1830] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:48:32 2022] PUT /identity/v3/projects/d598fdd196454ba2a4defaaa7e674f0e/users/79c0cf5642504370b3f8897ba4d4cfb6/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:32.864821 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8f50b-cd24-44cf-944d-5e085747c78d None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:32.864821 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8f50b-cd24-44cf-944d-5e085747c78d None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:32.865092 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8f50b-cd24-44cf-944d-5e085747c78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:32.933155 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-e4d8f50b-cd24-44cf-944d-5e085747c78d None None] MFA Rules not processed for user `79c0cf5642504370b3f8897ba4d4cfb6`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:32.934963 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e4d8f50b-cd24-44cf-944d-5e085747c78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:32.940476 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae37f13-ba76-4799-8f41-16d435c6507b admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:33.093847 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae37f13-ba76-4799-8f41-16d435c6507b admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:33.104148 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19bdd2-bc3c-44f2-933b-c2dcf19ca806 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:33.105492 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19bdd2-bc3c-44f2-933b-c2dcf19ca806 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:33.108120 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19bdd2-bc3c-44f2-933b-c2dcf19ca806 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:33.108672 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19bdd2-bc3c-44f2-933b-c2dcf19ca806 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:33.115880 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 915/1831] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:32 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:33.127750 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8f50b-cd24-44cf-944d-5e085747c78d None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:33.129329 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e4d8f50b-cd24-44cf-944d-5e085747c78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:33.145473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8f50b-cd24-44cf-944d-5e085747c78d None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:33.149751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e815d9b1-1d46-45a2-a228-1724c57cd603 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:33.150711 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815d9b1-1d46-45a2-a228-1724c57cd603 service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:33.151083 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815d9b1-1d46-45a2-a228-1724c57cd603 service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:33.151433 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815d9b1-1d46-45a2-a228-1724c57cd603 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:33.163419 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e815d9b1-1d46-45a2-a228-1724c57cd603 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:33.165542 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e815d9b1-1d46-45a2-a228-1724c57cd603 service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:33.178249 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 916/1832] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:48:33 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:48:33.263454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19bdd2-bc3c-44f2-933b-c2dcf19ca806 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:33.265554 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19bdd2-bc3c-44f2-933b-c2dcf19ca806 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:33.282802 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 917/1833] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:48:32 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 14:48:33.800597 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e815d9b1-1d46-45a2-a228-1724c57cd603 service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:33.864080 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:33.927725 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e815d9b1-1d46-45a2-a228-1724c57cd603 service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:33.933488 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c7942-f200-48e5-a729-826362d14676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:33.934442 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c7942-f200-48e5-a729-826362d14676 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:33.934755 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c7942-f200-48e5-a729-826362d14676 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:33.935038 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c7942-f200-48e5-a729-826362d14676 admin admin] PATH_INFO: `/v3/users/2b6922bd46824c5dbf5dd2aa91215efc` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:33.936484 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c7942-f200-48e5-a729-826362d14676 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b6922bd46824c5dbf5dd2aa91215efc)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:33.938459 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c7942-f200-48e5-a729-826362d14676 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:33.988273 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ac7c7942-f200-48e5-a729-826362d14676 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b6922bd46824c5dbf5dd2aa91215efc'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:34.002334 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ac7c7942-f200-48e5-a729-826362d14676 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b6922bd46824c5dbf5dd2aa91215efc'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:34.003303 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ac7c7942-f200-48e5-a729-826362d14676 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b6922bd46824c5dbf5dd2aa91215efc'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:34.004044 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ac7c7942-f200-48e5-a729-826362d14676 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b6922bd46824c5dbf5dd2aa91215efc'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:34.012779 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ac7c7942-f200-48e5-a729-826362d14676 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b6922bd46824c5dbf5dd2aa91215efc'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:34.029748 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 917/1834] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:48:33 2022] DELETE /identity/v3/users/2b6922bd46824c5dbf5dd2aa91215efc => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:34.036508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b041659-284b-4ca6-9df9-c535ce3d4960 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:34.037265 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b041659-284b-4ca6-9df9-c535ce3d4960 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:34.037265 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b041659-284b-4ca6-9df9-c535ce3d4960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:34.056718 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-6b041659-284b-4ca6-9df9-c535ce3d4960 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:34.057947 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-6b041659-284b-4ca6-9df9-c535ce3d4960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:34.122673 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 918/1835] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:34.132806 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c7942-f200-48e5-a729-826362d14676 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:34.134491 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-ac7c7942-f200-48e5-a729-826362d14676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:34.200657 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c7942-f200-48e5-a729-826362d14676 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:34.204684 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d37f3d5-a8c6-4dab-aa69-9ea959bdaf91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:34.205368 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37f3d5-a8c6-4dab-aa69-9ea959bdaf91 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:34.205660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37f3d5-a8c6-4dab-aa69-9ea959bdaf91 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:34.205940 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37f3d5-a8c6-4dab-aa69-9ea959bdaf91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:34.216393 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d37f3d5-a8c6-4dab-aa69-9ea959bdaf91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:34.217878 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d37f3d5-a8c6-4dab-aa69-9ea959bdaf91 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:34.229092 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 918/1836] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:48:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:48:34.757280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b041659-284b-4ca6-9df9-c535ce3d4960 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:34.759559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-6b041659-284b-4ca6-9df9-c535ce3d4960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:34.768800 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:34.777321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b041659-284b-4ca6-9df9-c535ce3d4960 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:34.787165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205c10d-0771-4744-977d-537b6b3da441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:34.788260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205c10d-0771-4744-977d-537b6b3da441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:34.795020 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205c10d-0771-4744-977d-537b6b3da441 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:34.797723 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205c10d-0771-4744-977d-537b6b3da441 admin admin] PATH_INFO: `/v3/projects/84d6eeddac2f47a7b84ed2e4c48cd8dd` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:34.805013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205c10d-0771-4744-977d-537b6b3da441 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84d6eeddac2f47a7b84ed2e4c48cd8dd)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:34.808808 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205c10d-0771-4744-977d-537b6b3da441 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:34.869246 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-0205c10d-0771-4744-977d-537b6b3da441 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84d6eeddac2f47a7b84ed2e4c48cd8dd', 'request_id': 'req-0205c10d-0771-4744-977d-537b6b3da441'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:34.876094 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-0205c10d-0771-4744-977d-537b6b3da441 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84d6eeddac2f47a7b84ed2e4c48cd8dd', 'request_id': 'req-0205c10d-0771-4744-977d-537b6b3da441'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:34.884903 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-0205c10d-0771-4744-977d-537b6b3da441 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84d6eeddac2f47a7b84ed2e4c48cd8dd', 'request_id': 'req-0205c10d-0771-4744-977d-537b6b3da441'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:34.899222 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-0205c10d-0771-4744-977d-537b6b3da441 admin admin] The token cache is being invalidate because project 84d6eeddac2f47a7b84ed2e4c48cd8dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:48:34.899516 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-0205c10d-0771-4744-977d-537b6b3da441 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:34.900199 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-0205c10d-0771-4744-977d-537b6b3da441 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:34.902096 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 919/1837] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:48:34 2022] DELETE /identity/v3/projects/84d6eeddac2f47a7b84ed2e4c48cd8dd => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:34.906022 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d37f3d5-a8c6-4dab-aa69-9ea959bdaf91 service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:34.919082 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d37f3d5-a8c6-4dab-aa69-9ea959bdaf91 service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:35.008436 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:35.020954 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0be2c4-bf64-4687-a43a-1e7458d01ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:35.022096 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0be2c4-bf64-4687-a43a-1e7458d01ca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:35.022448 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0be2c4-bf64-4687-a43a-1e7458d01ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:35.023066 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0be2c4-bf64-4687-a43a-1e7458d01ca3 admin admin] PATH_INFO: `/v3/users/79c0cf5642504370b3f8897ba4d4cfb6` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:35.024437 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0be2c4-bf64-4687-a43a-1e7458d01ca3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79c0cf5642504370b3f8897ba4d4cfb6)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:35.026685 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0be2c4-bf64-4687-a43a-1e7458d01ca3 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:35.081110 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-2a0be2c4-bf64-4687-a43a-1e7458d01ca3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79c0cf5642504370b3f8897ba4d4cfb6'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:35.098756 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-2a0be2c4-bf64-4687-a43a-1e7458d01ca3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79c0cf5642504370b3f8897ba4d4cfb6'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:35.098756 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-2a0be2c4-bf64-4687-a43a-1e7458d01ca3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79c0cf5642504370b3f8897ba4d4cfb6'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:35.098756 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-2a0be2c4-bf64-4687-a43a-1e7458d01ca3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79c0cf5642504370b3f8897ba4d4cfb6'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:35.106399 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-2a0be2c4-bf64-4687-a43a-1e7458d01ca3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79c0cf5642504370b3f8897ba4d4cfb6'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:35.123949 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 919/1838] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:48:34 2022] DELETE /identity/v3/users/79c0cf5642504370b3f8897ba4d4cfb6 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:35.462998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205c10d-0771-4744-977d-537b6b3da441 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:35.464595 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-0205c10d-0771-4744-977d-537b6b3da441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:35.479269 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205c10d-0771-4744-977d-537b6b3da441 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:35.549345 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad93507-4529-4422-8f46-bacd8ca6ff62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:35.550962 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad93507-4529-4422-8f46-bacd8ca6ff62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:35.551294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad93507-4529-4422-8f46-bacd8ca6ff62 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:35.551736 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad93507-4529-4422-8f46-bacd8ca6ff62 admin admin] PATH_INFO: `/v3/projects/d598fdd196454ba2a4defaaa7e674f0e` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:35.553131 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad93507-4529-4422-8f46-bacd8ca6ff62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d598fdd196454ba2a4defaaa7e674f0e)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:35.555501 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad93507-4529-4422-8f46-bacd8ca6ff62 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:35.638760 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-5ad93507-4529-4422-8f46-bacd8ca6ff62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd598fdd196454ba2a4defaaa7e674f0e', 'request_id': 'req-5ad93507-4529-4422-8f46-bacd8ca6ff62'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:35.654485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-5ad93507-4529-4422-8f46-bacd8ca6ff62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd598fdd196454ba2a4defaaa7e674f0e', 'request_id': 'req-5ad93507-4529-4422-8f46-bacd8ca6ff62'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:35.668058 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-5ad93507-4529-4422-8f46-bacd8ca6ff62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd598fdd196454ba2a4defaaa7e674f0e', 'request_id': 'req-5ad93507-4529-4422-8f46-bacd8ca6ff62'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:35.683035 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-5ad93507-4529-4422-8f46-bacd8ca6ff62 admin admin] The token cache is being invalidate because project d598fdd196454ba2a4defaaa7e674f0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:48:35.683393 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-5ad93507-4529-4422-8f46-bacd8ca6ff62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:35.684085 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-5ad93507-4529-4422-8f46-bacd8ca6ff62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:35.686861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 920/1839] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:48:35 2022] DELETE /identity/v3/projects/d598fdd196454ba2a4defaaa7e674f0e => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:35.698538 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ad822-b3f3-4b9f-97ab-6e25785bb432 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:35.698833 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ad822-b3f3-4b9f-97ab-6e25785bb432 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:35.699144 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ad822-b3f3-4b9f-97ab-6e25785bb432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:35.735840 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-9b6ad822-b3f3-4b9f-97ab-6e25785bb432 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:35.738110 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-9b6ad822-b3f3-4b9f-97ab-6e25785bb432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:35.855341 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 920/1840] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:35.857158 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee33b72-9a6c-49fb-83f6-1e2986620bca None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:35.857453 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee33b72-9a6c-49fb-83f6-1e2986620bca None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:35.857737 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee33b72-9a6c-49fb-83f6-1e2986620bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:35.878726 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-aee33b72-9a6c-49fb-83f6-1e2986620bca None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:35.880033 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-aee33b72-9a6c-49fb-83f6-1e2986620bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:35.889764 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 921/1841] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:35.893674 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ad822-b3f3-4b9f-97ab-6e25785bb432 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:35.904330 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ad822-b3f3-4b9f-97ab-6e25785bb432 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:35.910898 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971e5668-7c47-454c-9956-13df2f2c2314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:35.911736 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e5668-7c47-454c-9956-13df2f2c2314 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:35.912088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e5668-7c47-454c-9956-13df2f2c2314 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:35.912435 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e5668-7c47-454c-9956-13df2f2c2314 admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:35.913408 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971e5668-7c47-454c-9956-13df2f2c2314 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:48:35.913649 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971e5668-7c47-454c-9956-13df2f2c2314 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:35.915446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971e5668-7c47-454c-9956-13df2f2c2314 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:35.930957 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 921/1842] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:48:35.935661 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee33b72-9a6c-49fb-83f6-1e2986620bca None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:35.949611 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee33b72-9a6c-49fb-83f6-1e2986620bca None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:35.955384 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367b452-2b02-4357-bc75-aed667e9caaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:35.956095 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367b452-2b02-4357-bc75-aed667e9caaa admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:35.956421 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367b452-2b02-4357-bc75-aed667e9caaa admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:35.956686 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367b452-2b02-4357-bc75-aed667e9caaa admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:35.957472 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e367b452-2b02-4357-bc75-aed667e9caaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:35.961840 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e367b452-2b02-4357-bc75-aed667e9caaa admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:35.972429 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:36.004742 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 922/1843] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:48:35 2022] POST /identity/v3/projects => generated 383 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:48:36.005837 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971e5668-7c47-454c-9956-13df2f2c2314 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:36.028464 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971e5668-7c47-454c-9956-13df2f2c2314 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:36.136976 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b35fb9-f22c-4a35-a660-b1c19f5ac4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:36.138496 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b35fb9-f22c-4a35-a660-b1c19f5ac4f9 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:36.141081 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b35fb9-f22c-4a35-a660-b1c19f5ac4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:36.141590 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b35fb9-f22c-4a35-a660-b1c19f5ac4f9 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:36.143624 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b35fb9-f22c-4a35-a660-b1c19f5ac4f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:36.146577 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b35fb9-f22c-4a35-a660-b1c19f5ac4f9 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:36.158144 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:36.181126 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 922/1844] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:48:36 2022] POST /identity/v3/users => generated 292 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:48:36.185465 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367b452-2b02-4357-bc75-aed667e9caaa admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:36.197070 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367b452-2b02-4357-bc75-aed667e9caaa admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:36.201892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a2223-851b-4fe9-ab24-eb05a8979a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:36.202596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a2223-851b-4fe9-ab24-eb05a8979a91 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:36.202880 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a2223-851b-4fe9-ab24-eb05a8979a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:36.203260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a2223-851b-4fe9-ab24-eb05a8979a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:36.203981 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a2223-851b-4fe9-ab24-eb05a8979a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:36.205639 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a2223-851b-4fe9-ab24-eb05a8979a91 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:36.220273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 923/1845] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:48:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:48:36.222500 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b35fb9-f22c-4a35-a660-b1c19f5ac4f9 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:36.236137 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b35fb9-f22c-4a35-a660-b1c19f5ac4f9 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:36.242414 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4d5b2-a6e5-402f-b235-0982fc4fdd9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:36.243232 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4d5b2-a6e5-402f-b235-0982fc4fdd9e admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:36.243556 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4d5b2-a6e5-402f-b235-0982fc4fdd9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:36.243830 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4d5b2-a6e5-402f-b235-0982fc4fdd9e admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:36.244647 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa4d5b2-a6e5-402f-b235-0982fc4fdd9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:36.247015 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa4d5b2-a6e5-402f-b235-0982fc4fdd9e admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:36.261665 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 923/1846] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:48:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:48:36.266916 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a2223-851b-4fe9-ab24-eb05a8979a91 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:36.278927 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a2223-851b-4fe9-ab24-eb05a8979a91 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:36.284937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6796c2-66ce-481f-95a5-d4b9bde9360e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:36.285959 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6796c2-66ce-481f-95a5-d4b9bde9360e admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:36.286283 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6796c2-66ce-481f-95a5-d4b9bde9360e admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:36.286600 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6796c2-66ce-481f-95a5-d4b9bde9360e admin admin] PATH_INFO: `/v3/projects/d13bcf27eb8140098a92a5b70b6b8c60/users/837a79e973944a8990d61205c9cf7b2b/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:36.308791 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6796c2-66ce-481f-95a5-d4b9bde9360e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d13bcf27eb8140098a92a5b70b6b8c60, user_id=837a79e973944a8990d61205c9cf7b2b, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:36.312306 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6796c2-66ce-481f-95a5-d4b9bde9360e admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:36.339668 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 924/1847] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:48:36 2022] PUT /identity/v3/projects/d13bcf27eb8140098a92a5b70b6b8c60/users/837a79e973944a8990d61205c9cf7b2b/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:36.351685 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f62ce-c5b9-46ed-ad88-1a22bb6dbedc None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:36.352065 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f62ce-c5b9-46ed-ad88-1a22bb6dbedc None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:36.352353 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f62ce-c5b9-46ed-ad88-1a22bb6dbedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:36.401891 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-6c7f62ce-c5b9-46ed-ad88-1a22bb6dbedc None None] MFA Rules not processed for user `837a79e973944a8990d61205c9cf7b2b`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:36.403963 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-6c7f62ce-c5b9-46ed-ad88-1a22bb6dbedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:36.542721 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 924/1848] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:36.559113 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6796c2-66ce-481f-95a5-d4b9bde9360e admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:36.566438 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-ba6796c2-66ce-481f-95a5-d4b9bde9360e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:36.663024 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6796c2-66ce-481f-95a5-d4b9bde9360e admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:36.670368 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d90cbd-6ee9-4512-92b5-5debe48dfb1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:36.672614 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d90cbd-6ee9-4512-92b5-5debe48dfb1d service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:36.673308 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d90cbd-6ee9-4512-92b5-5debe48dfb1d service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:36.673965 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d90cbd-6ee9-4512-92b5-5debe48dfb1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:36.691566 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d90cbd-6ee9-4512-92b5-5debe48dfb1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:36.694479 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d90cbd-6ee9-4512-92b5-5debe48dfb1d service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:36.708444 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 925/1849] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:48:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 14:48:36.731768 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f62ce-c5b9-46ed-ad88-1a22bb6dbedc None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:36.813083 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f62ce-c5b9-46ed-ad88-1a22bb6dbedc None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:36.818409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689b6766-eaaf-4948-97eb-d2d82f4942b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:36.819356 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b6766-eaaf-4948-97eb-d2d82f4942b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:36.819656 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b6766-eaaf-4948-97eb-d2d82f4942b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:36.819913 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b6766-eaaf-4948-97eb-d2d82f4942b0 admin admin] PATH_INFO: `/v3/users/837a79e973944a8990d61205c9cf7b2b` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:36.821266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689b6766-eaaf-4948-97eb-d2d82f4942b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=837a79e973944a8990d61205c9cf7b2b)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:36.823297 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689b6766-eaaf-4948-97eb-d2d82f4942b0 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:36.876557 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-689b6766-eaaf-4948-97eb-d2d82f4942b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '837a79e973944a8990d61205c9cf7b2b'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:36.891007 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-689b6766-eaaf-4948-97eb-d2d82f4942b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '837a79e973944a8990d61205c9cf7b2b'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:36.891862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-689b6766-eaaf-4948-97eb-d2d82f4942b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '837a79e973944a8990d61205c9cf7b2b'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:36.892557 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-689b6766-eaaf-4948-97eb-d2d82f4942b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '837a79e973944a8990d61205c9cf7b2b'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:36.897589 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:DELETE=9 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:36.901350 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-689b6766-eaaf-4948-97eb-d2d82f4942b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '837a79e973944a8990d61205c9cf7b2b'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:36.920244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 925/1850] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:48:36 2022] DELETE /identity/v3/users/837a79e973944a8990d61205c9cf7b2b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:36.924665 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed01a75-2207-4596-80ca-8378782769c6 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:36.924960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed01a75-2207-4596-80ca-8378782769c6 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:36.925244 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed01a75-2207-4596-80ca-8378782769c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:36.943235 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-5ed01a75-2207-4596-80ca-8378782769c6 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:36.944651 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-5ed01a75-2207-4596-80ca-8378782769c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:37.016927 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 926/1851] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:37.027467 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689b6766-eaaf-4948-97eb-d2d82f4942b0 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:37.029308 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-689b6766-eaaf-4948-97eb-d2d82f4942b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:37.097501 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689b6766-eaaf-4948-97eb-d2d82f4942b0 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:37.102638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac50d4-570a-44c1-bdbe-0dc98b57ec34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:37.104289 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac50d4-570a-44c1-bdbe-0dc98b57ec34 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:37.104762 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac50d4-570a-44c1-bdbe-0dc98b57ec34 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:37.104896 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac50d4-570a-44c1-bdbe-0dc98b57ec34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:37.124158 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac50d4-570a-44c1-bdbe-0dc98b57ec34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:37.124158 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac50d4-570a-44c1-bdbe-0dc98b57ec34 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:37.139360 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 926/1852] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:48:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:48:37.718914 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed01a75-2207-4596-80ca-8378782769c6 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:37.721458 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-5ed01a75-2207-4596-80ca-8378782769c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:37.747901 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed01a75-2207-4596-80ca-8378782769c6 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:37.754269 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49505e56-2d9a-4693-a83c-2a5ba85b1f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:37.760312 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49505e56-2d9a-4693-a83c-2a5ba85b1f25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:37.761161 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49505e56-2d9a-4693-a83c-2a5ba85b1f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:37.761630 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49505e56-2d9a-4693-a83c-2a5ba85b1f25 admin admin] PATH_INFO: `/v3/projects/d13bcf27eb8140098a92a5b70b6b8c60` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:37.768491 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49505e56-2d9a-4693-a83c-2a5ba85b1f25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d13bcf27eb8140098a92a5b70b6b8c60)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:37.779039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49505e56-2d9a-4693-a83c-2a5ba85b1f25 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:37.850667 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-49505e56-2d9a-4693-a83c-2a5ba85b1f25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd13bcf27eb8140098a92a5b70b6b8c60', 'request_id': 'req-49505e56-2d9a-4693-a83c-2a5ba85b1f25'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:37.851974 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:37.858562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-49505e56-2d9a-4693-a83c-2a5ba85b1f25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd13bcf27eb8140098a92a5b70b6b8c60', 'request_id': 'req-49505e56-2d9a-4693-a83c-2a5ba85b1f25'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:37.867110 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-49505e56-2d9a-4693-a83c-2a5ba85b1f25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd13bcf27eb8140098a92a5b70b6b8c60', 'request_id': 'req-49505e56-2d9a-4693-a83c-2a5ba85b1f25'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:37.879649 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-49505e56-2d9a-4693-a83c-2a5ba85b1f25 admin admin] The token cache is being invalidate because project d13bcf27eb8140098a92a5b70b6b8c60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:48:37.879948 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-49505e56-2d9a-4693-a83c-2a5ba85b1f25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:37.880650 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-49505e56-2d9a-4693-a83c-2a5ba85b1f25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:37.882323 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 927/1853] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:48:37 2022] DELETE /identity/v3/projects/d13bcf27eb8140098a92a5b70b6b8c60 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:37.887996 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5919e7-a1f1-4d18-89c7-72829fd58ac5 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:37.888286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5919e7-a1f1-4d18-89c7-72829fd58ac5 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:37.888600 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5919e7-a1f1-4d18-89c7-72829fd58ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:37.908029 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-8e5919e7-a1f1-4d18-89c7-72829fd58ac5 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:37.909330 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-8e5919e7-a1f1-4d18-89c7-72829fd58ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:37.981536 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 927/1854] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:48:37.990610 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b675bd-b147-4efa-a8e1-4972a183c48b None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:37.990948 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b675bd-b147-4efa-a8e1-4972a183c48b None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:37.991275 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b675bd-b147-4efa-a8e1-4972a183c48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:38.011674 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-e5b675bd-b147-4efa-a8e1-4972a183c48b None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:38.013095 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-e5b675bd-b147-4efa-a8e1-4972a183c48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:38.023782 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 928/1855] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:38.028711 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5919e7-a1f1-4d18-89c7-72829fd58ac5 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:38.040295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5919e7-a1f1-4d18-89c7-72829fd58ac5 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:38.048155 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59131c50-7fd7-4c34-87f7-065bf6fc1679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:38.049331 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59131c50-7fd7-4c34-87f7-065bf6fc1679 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:38.049331 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59131c50-7fd7-4c34-87f7-065bf6fc1679 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:38.049331 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59131c50-7fd7-4c34-87f7-065bf6fc1679 admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:38.050245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59131c50-7fd7-4c34-87f7-065bf6fc1679 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:48:38.050570 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59131c50-7fd7-4c34-87f7-065bf6fc1679 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:38.052381 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59131c50-7fd7-4c34-87f7-065bf6fc1679 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:38.067021 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 928/1856] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:48:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:48:38.072085 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b675bd-b147-4efa-a8e1-4972a183c48b None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:38.091434 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b675bd-b147-4efa-a8e1-4972a183c48b None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:38.097279 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db155e-f1dc-4b6e-a6bb-5b301a909d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:38.098015 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db155e-f1dc-4b6e-a6bb-5b301a909d3d admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:38.098347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db155e-f1dc-4b6e-a6bb-5b301a909d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:38.098621 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db155e-f1dc-4b6e-a6bb-5b301a909d3d admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:38.099395 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4db155e-f1dc-4b6e-a6bb-5b301a909d3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:38.101454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4db155e-f1dc-4b6e-a6bb-5b301a909d3d admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:38.131532 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 929/1857] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:48:38 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:48:38.135920 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59131c50-7fd7-4c34-87f7-065bf6fc1679 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:38.147654 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59131c50-7fd7-4c34-87f7-065bf6fc1679 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:38.216106 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd0024-d835-4207-b1f7-a645f61b6950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:38.216755 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd0024-d835-4207-b1f7-a645f61b6950 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:38.219472 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd0024-d835-4207-b1f7-a645f61b6950 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:38.219472 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd0024-d835-4207-b1f7-a645f61b6950 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:38.219472 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd0024-d835-4207-b1f7-a645f61b6950 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:38.220065 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd0024-d835-4207-b1f7-a645f61b6950 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:38.248353 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 929/1858] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:48:38 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:48:38.252906 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db155e-f1dc-4b6e-a6bb-5b301a909d3d admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:38.264527 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db155e-f1dc-4b6e-a6bb-5b301a909d3d admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:38.269439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edde62b-2ba5-4085-83ca-b1f1c8864080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:38.270198 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edde62b-2ba5-4085-83ca-b1f1c8864080 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:38.270476 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edde62b-2ba5-4085-83ca-b1f1c8864080 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:38.270725 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edde62b-2ba5-4085-83ca-b1f1c8864080 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:38.271543 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edde62b-2ba5-4085-83ca-b1f1c8864080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:38.273217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edde62b-2ba5-4085-83ca-b1f1c8864080 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:38.285552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 930/1859] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:48:38.292238 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd0024-d835-4207-b1f7-a645f61b6950 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:38.307016 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd0024-d835-4207-b1f7-a645f61b6950 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:38.312510 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec776840-6a1d-4e82-a4dd-0345ae3d6424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:38.313270 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec776840-6a1d-4e82-a4dd-0345ae3d6424 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:38.313571 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec776840-6a1d-4e82-a4dd-0345ae3d6424 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:38.313859 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec776840-6a1d-4e82-a4dd-0345ae3d6424 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:38.314723 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec776840-6a1d-4e82-a4dd-0345ae3d6424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:38.316772 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec776840-6a1d-4e82-a4dd-0345ae3d6424 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:38.318443 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:38.332053 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 930/1860] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:48:38.336300 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edde62b-2ba5-4085-83ca-b1f1c8864080 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:38.352584 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edde62b-2ba5-4085-83ca-b1f1c8864080 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:38.359534 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f028cf-1ed7-4379-b2e7-ac6f63a5fb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:38.360515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f028cf-1ed7-4379-b2e7-ac6f63a5fb94 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:38.360847 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f028cf-1ed7-4379-b2e7-ac6f63a5fb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:38.361139 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f028cf-1ed7-4379-b2e7-ac6f63a5fb94 admin admin] PATH_INFO: `/v3/projects/2c8531e2e07a47eaae4c66d0b82df2d9/users/dd610b017e7d4bd786db6bcd08e60722/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:38.380783 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f028cf-1ed7-4379-b2e7-ac6f63a5fb94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c8531e2e07a47eaae4c66d0b82df2d9, user_id=dd610b017e7d4bd786db6bcd08e60722, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:38.383719 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f028cf-1ed7-4379-b2e7-ac6f63a5fb94 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:38.401212 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 931/1861] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:48:38 2022] PUT /identity/v3/projects/2c8531e2e07a47eaae4c66d0b82df2d9/users/dd610b017e7d4bd786db6bcd08e60722/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 14:48:38.407799 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257be5e-bf19-4daf-9248-2f9a4a020d8d None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:38.408150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257be5e-bf19-4daf-9248-2f9a4a020d8d None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:38.409733 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257be5e-bf19-4daf-9248-2f9a4a020d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:38.468784 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-2257be5e-bf19-4daf-9248-2f9a4a020d8d None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:38.471330 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-2257be5e-bf19-4daf-9248-2f9a4a020d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:38.564434 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 931/1862] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:48:38.575342 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f028cf-1ed7-4379-b2e7-ac6f63a5fb94 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:38.577387 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-d9f028cf-1ed7-4379-b2e7-ac6f63a5fb94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:38.661250 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f028cf-1ed7-4379-b2e7-ac6f63a5fb94 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:38.667414 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b5d6dc-d2c3-40cd-ada3-582b9ee61001 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:38.668122 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5d6dc-d2c3-40cd-ada3-582b9ee61001 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:38.668419 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5d6dc-d2c3-40cd-ada3-582b9ee61001 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:38.668706 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5d6dc-d2c3-40cd-ada3-582b9ee61001 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:38.679534 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b5d6dc-d2c3-40cd-ada3-582b9ee61001 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:38.681014 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b5d6dc-d2c3-40cd-ada3-582b9ee61001 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:38.692448 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 932/1863] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:48:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:48:43.580961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3a6d0-a90e-43a1-beca-eebd8b862bad None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:43.581552 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3a6d0-a90e-43a1-beca-eebd8b862bad None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:43.582256 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3a6d0-a90e-43a1-beca-eebd8b862bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:43.612301 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-17f3a6d0-a90e-43a1-beca-eebd8b862bad None None] MFA Rules not processed for user `dd610b017e7d4bd786db6bcd08e60722`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:43.613570 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-17f3a6d0-a90e-43a1-beca-eebd8b862bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:43.698342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 932/1864] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:43.703544 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b5d6dc-d2c3-40cd-ada3-582b9ee61001 service neutron] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:43.714742 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b5d6dc-d2c3-40cd-ada3-582b9ee61001 service neutron] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:43.734739 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e6541-80ab-4881-ae8f-076db1b38607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:43.734739 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e6541-80ab-4881-ae8f-076db1b38607 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:43.734739 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e6541-80ab-4881-ae8f-076db1b38607 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:43.734739 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e6541-80ab-4881-ae8f-076db1b38607 admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:43.735533 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434e6541-80ab-4881-ae8f-076db1b38607 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:43.735533 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434e6541-80ab-4881-ae8f-076db1b38607 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:43.791909 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 933/1865] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:48:43 2022] POST /identity/v3/projects => generated 379 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:48:43.796236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3a6d0-a90e-43a1-beca-eebd8b862bad None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:43.816978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3a6d0-a90e-43a1-beca-eebd8b862bad None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:43.957354 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5849e1d6-e8bb-4340-83b7-1cb000ef8631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:43.958146 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849e1d6-e8bb-4340-83b7-1cb000ef8631 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:43.958431 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849e1d6-e8bb-4340-83b7-1cb000ef8631 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:43.958730 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849e1d6-e8bb-4340-83b7-1cb000ef8631 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:43.959514 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5849e1d6-e8bb-4340-83b7-1cb000ef8631 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:43.961514 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5849e1d6-e8bb-4340-83b7-1cb000ef8631 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:43.993205 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 933/1866] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:48:43 2022] POST /identity/v3/users => generated 296 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:48:43.994710 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e6541-80ab-4881-ae8f-076db1b38607 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:44.014054 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e6541-80ab-4881-ae8f-076db1b38607 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:44.036880 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5453d96-9b98-4bf1-80f5-ae93b60f3296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:44.037791 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5453d96-9b98-4bf1-80f5-ae93b60f3296 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:44.038220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5453d96-9b98-4bf1-80f5-ae93b60f3296 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:44.039444 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5453d96-9b98-4bf1-80f5-ae93b60f3296 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:44.040356 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5453d96-9b98-4bf1-80f5-ae93b60f3296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:44.042260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5453d96-9b98-4bf1-80f5-ae93b60f3296 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:44.056928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 934/1867] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:48:44.060843 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5849e1d6-e8bb-4340-83b7-1cb000ef8631 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:44.073473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5849e1d6-e8bb-4340-83b7-1cb000ef8631 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:44.080756 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ace2a-eb27-4a06-8d65-58ad6a2ee1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:44.081828 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ace2a-eb27-4a06-8d65-58ad6a2ee1fa admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:44.082242 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ace2a-eb27-4a06-8d65-58ad6a2ee1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:44.082640 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ace2a-eb27-4a06-8d65-58ad6a2ee1fa admin admin] PATH_INFO: `/v3/projects/1cc85169a9644cd28b5cd790efac0680/users/513b043962cf48c695c7c724bd08395f/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:44.102685 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:44.109091 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3ace2a-eb27-4a06-8d65-58ad6a2ee1fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cc85169a9644cd28b5cd790efac0680, user_id=513b043962cf48c695c7c724bd08395f, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:44.112710 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3ace2a-eb27-4a06-8d65-58ad6a2ee1fa admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:44.147985 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 934/1868] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:48:44 2022] PUT /identity/v3/projects/1cc85169a9644cd28b5cd790efac0680/users/513b043962cf48c695c7c724bd08395f/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:44.152664 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5453d96-9b98-4bf1-80f5-ae93b60f3296 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:44.170103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5453d96-9b98-4bf1-80f5-ae93b60f3296 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:44.295894 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:44.333771 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5fb0d-97ae-40ae-a546-64a98aed24f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:44.334494 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5fb0d-97ae-40ae-a546-64a98aed24f1 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:44.334761 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5fb0d-97ae-40ae-a546-64a98aed24f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:44.335032 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5fb0d-97ae-40ae-a546-64a98aed24f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:44.335802 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d5fb0d-97ae-40ae-a546-64a98aed24f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:44.337450 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d5fb0d-97ae-40ae-a546-64a98aed24f1 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:44.359876 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 935/1869] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:48:44.364159 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ace2a-eb27-4a06-8d65-58ad6a2ee1fa admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:44.378245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ace2a-eb27-4a06-8d65-58ad6a2ee1fa admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:44.383459 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ac1e1-b30d-4fc4-835f-2546543340fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:44.384747 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ac1e1-b30d-4fc4-835f-2546543340fd admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:44.385272 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ac1e1-b30d-4fc4-835f-2546543340fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:44.385724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ac1e1-b30d-4fc4-835f-2546543340fd admin admin] PATH_INFO: `/v3/projects/1cc85169a9644cd28b5cd790efac0680/users/513b043962cf48c695c7c724bd08395f/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:44.392173 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ac1e1-b30d-4fc4-835f-2546543340fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cc85169a9644cd28b5cd790efac0680, user_id=513b043962cf48c695c7c724bd08395f, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:44.395879 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ac1e1-b30d-4fc4-835f-2546543340fd admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:44.427526 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 935/1870] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:48:44 2022] PUT /identity/v3/projects/1cc85169a9644cd28b5cd790efac0680/users/513b043962cf48c695c7c724bd08395f/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:47.211942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5fb0d-97ae-40ae-a546-64a98aed24f1 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:47.214697 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-f3d5fb0d-97ae-40ae-a546-64a98aed24f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:47.230836 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5fb0d-97ae-40ae-a546-64a98aed24f1 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:47.318539 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de388b99-43e3-4af7-8aa6-409f40855c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:47.318807 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de388b99-43e3-4af7-8aa6-409f40855c4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:47.319165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de388b99-43e3-4af7-8aa6-409f40855c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:47.319455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de388b99-43e3-4af7-8aa6-409f40855c4e admin admin] PATH_INFO: `/v3/users/45a72ac2c45e423891dd78a20b84d52f` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:47.320778 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de388b99-43e3-4af7-8aa6-409f40855c4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=45a72ac2c45e423891dd78a20b84d52f)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:47.322730 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de388b99-43e3-4af7-8aa6-409f40855c4e admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:47.377883 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-de388b99-43e3-4af7-8aa6-409f40855c4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45a72ac2c45e423891dd78a20b84d52f'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:47.392736 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-de388b99-43e3-4af7-8aa6-409f40855c4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45a72ac2c45e423891dd78a20b84d52f'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:47.393575 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-de388b99-43e3-4af7-8aa6-409f40855c4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45a72ac2c45e423891dd78a20b84d52f'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:47.394182 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-de388b99-43e3-4af7-8aa6-409f40855c4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45a72ac2c45e423891dd78a20b84d52f'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:47.403611 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-de388b99-43e3-4af7-8aa6-409f40855c4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45a72ac2c45e423891dd78a20b84d52f'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:47.428591 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 936/1871] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:48:47 2022] DELETE /identity/v3/users/45a72ac2c45e423891dd78a20b84d52f => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:47.434793 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00480d8-6206-44c6-9765-c077ab8c2660 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:47.435428 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00480d8-6206-44c6-9765-c077ab8c2660 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:47.435428 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00480d8-6206-44c6-9765-c077ab8c2660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:47.462381 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-e00480d8-6206-44c6-9765-c077ab8c2660 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:47.463682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e00480d8-6206-44c6-9765-c077ab8c2660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:47.552880 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 936/1872] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:48:47.570558 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de388b99-43e3-4af7-8aa6-409f40855c4e admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:47.573524 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-de388b99-43e3-4af7-8aa6-409f40855c4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:47.716180 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de388b99-43e3-4af7-8aa6-409f40855c4e admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:47.722204 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff95e28-b10c-46e2-b24f-b7da8de8834a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:47.723267 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff95e28-b10c-46e2-b24f-b7da8de8834a service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:47.724321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff95e28-b10c-46e2-b24f-b7da8de8834a service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:47.725143 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff95e28-b10c-46e2-b24f-b7da8de8834a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:47.737130 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff95e28-b10c-46e2-b24f-b7da8de8834a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:47.739595 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff95e28-b10c-46e2-b24f-b7da8de8834a service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:47.754253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 937/1873] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:48:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 14:48:48.175232 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00480d8-6206-44c6-9765-c077ab8c2660 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:48.177839 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e00480d8-6206-44c6-9765-c077ab8c2660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:48.213654 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00480d8-6206-44c6-9765-c077ab8c2660 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:48.225450 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01908225-dbcf-4b29-b1b3-7762f0a6b783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:48.226590 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01908225-dbcf-4b29-b1b3-7762f0a6b783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:48.228005 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01908225-dbcf-4b29-b1b3-7762f0a6b783 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:48.228919 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01908225-dbcf-4b29-b1b3-7762f0a6b783 admin admin] PATH_INFO: `/v3/projects/78e39b55dfb4430f8ef1ccf553f6eae3` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:48.232033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01908225-dbcf-4b29-b1b3-7762f0a6b783 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78e39b55dfb4430f8ef1ccf553f6eae3)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:48.235247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01908225-dbcf-4b29-b1b3-7762f0a6b783 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:48.367364 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-01908225-dbcf-4b29-b1b3-7762f0a6b783 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78e39b55dfb4430f8ef1ccf553f6eae3', 'request_id': 'req-01908225-dbcf-4b29-b1b3-7762f0a6b783'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:48.392468 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-01908225-dbcf-4b29-b1b3-7762f0a6b783 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78e39b55dfb4430f8ef1ccf553f6eae3', 'request_id': 'req-01908225-dbcf-4b29-b1b3-7762f0a6b783'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:48.416386 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-01908225-dbcf-4b29-b1b3-7762f0a6b783 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78e39b55dfb4430f8ef1ccf553f6eae3', 'request_id': 'req-01908225-dbcf-4b29-b1b3-7762f0a6b783'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:48.451288 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-01908225-dbcf-4b29-b1b3-7762f0a6b783 admin admin] The token cache is being invalidate because project 78e39b55dfb4430f8ef1ccf553f6eae3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:48:48.452330 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-01908225-dbcf-4b29-b1b3-7762f0a6b783 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:48.454921 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-01908225-dbcf-4b29-b1b3-7762f0a6b783 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:48.459267 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 937/1874] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:48:48 2022] DELETE /identity/v3/projects/78e39b55dfb4430f8ef1ccf553f6eae3 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:48.464084 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff95e28-b10c-46e2-b24f-b7da8de8834a service neutron] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:48.475279 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:48.502457 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff95e28-b10c-46e2-b24f-b7da8de8834a service neutron] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:48.646121 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ed957-75f3-49d1-9271-abf8fe768db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:48.647993 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ed957-75f3-49d1-9271-abf8fe768db4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:48.648477 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ed957-75f3-49d1-9271-abf8fe768db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:48.648957 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ed957-75f3-49d1-9271-abf8fe768db4 admin admin] PATH_INFO: `/v3/users/5b99217fba44450791186bb8e6c7c54b` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:48.653009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ed957-75f3-49d1-9271-abf8fe768db4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b99217fba44450791186bb8e6c7c54b)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:48.657054 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ed957-75f3-49d1-9271-abf8fe768db4 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:48.822722 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fe7ed957-75f3-49d1-9271-abf8fe768db4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b99217fba44450791186bb8e6c7c54b'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:48.878298 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fe7ed957-75f3-49d1-9271-abf8fe768db4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b99217fba44450791186bb8e6c7c54b'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:48.879503 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fe7ed957-75f3-49d1-9271-abf8fe768db4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b99217fba44450791186bb8e6c7c54b'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:48.883542 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fe7ed957-75f3-49d1-9271-abf8fe768db4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b99217fba44450791186bb8e6c7c54b'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:48.895109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fe7ed957-75f3-49d1-9271-abf8fe768db4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b99217fba44450791186bb8e6c7c54b'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:48.916109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 938/1875] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:48:48 2022] DELETE /identity/v3/users/5b99217fba44450791186bb8e6c7c54b => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:50.528914 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68897e-b5f9-4bc4-a241-8de7196d8244 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:50.530133 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68897e-b5f9-4bc4-a241-8de7196d8244 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:50.530585 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68897e-b5f9-4bc4-a241-8de7196d8244 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:50.533082 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 938/1876] 10.222.0.15 () {54 vars in 921 bytes} [Wed Jul 20 14:48:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:48:50.579842 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd26b680-cde2-4f5e-974d-102906129026 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:50.580446 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd26b680-cde2-4f5e-974d-102906129026 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:50.581242 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd26b680-cde2-4f5e-974d-102906129026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:50.618744 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-cd26b680-cde2-4f5e-974d-102906129026 None None] MFA Rules not processed for user `5d4a107ed04348b59d20d1703dbe04eb`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:50.621145 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-cd26b680-cde2-4f5e-974d-102906129026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:50.809117 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 939/1877] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Jul 20 14:48:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:51.019576 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68897e-b5f9-4bc4-a241-8de7196d8244 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:51.023928 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-ac68897e-b5f9-4bc4-a241-8de7196d8244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:51.169616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68897e-b5f9-4bc4-a241-8de7196d8244 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:51.178160 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4d9a1-1482-4452-b4ec-fa9bb55f0c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:51.181326 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4d9a1-1482-4452-b4ec-fa9bb55f0c88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:51.182186 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4d9a1-1482-4452-b4ec-fa9bb55f0c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:51.182955 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4d9a1-1482-4452-b4ec-fa9bb55f0c88 admin admin] PATH_INFO: `/v3/projects/afc05851b4824b03ab71604b1e75cbb9` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:51.185811 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d4d9a1-1482-4452-b4ec-fa9bb55f0c88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afc05851b4824b03ab71604b1e75cbb9)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:51.190208 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d4d9a1-1482-4452-b4ec-fa9bb55f0c88 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:51.194862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:51.289300 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-36d4d9a1-1482-4452-b4ec-fa9bb55f0c88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afc05851b4824b03ab71604b1e75cbb9', 'request_id': 'req-36d4d9a1-1482-4452-b4ec-fa9bb55f0c88'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:51.296732 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-36d4d9a1-1482-4452-b4ec-fa9bb55f0c88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afc05851b4824b03ab71604b1e75cbb9', 'request_id': 'req-36d4d9a1-1482-4452-b4ec-fa9bb55f0c88'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:51.305187 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-36d4d9a1-1482-4452-b4ec-fa9bb55f0c88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afc05851b4824b03ab71604b1e75cbb9', 'request_id': 'req-36d4d9a1-1482-4452-b4ec-fa9bb55f0c88'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:51.323733 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-36d4d9a1-1482-4452-b4ec-fa9bb55f0c88 admin admin] The token cache is being invalidate because project afc05851b4824b03ab71604b1e75cbb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:48:51.324180 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-36d4d9a1-1482-4452-b4ec-fa9bb55f0c88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:51.324989 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-36d4d9a1-1482-4452-b4ec-fa9bb55f0c88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:48:51.326976 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 939/1878] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:48:51 2022] DELETE /identity/v3/projects/afc05851b4824b03ab71604b1e75cbb9 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:51.331997 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ae061-dcd8-4b15-bc34-4a95388d5f08 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:51.332383 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ae061-dcd8-4b15-bc34-4a95388d5f08 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:51.332749 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ae061-dcd8-4b15-bc34-4a95388d5f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:51.355415 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-8a0ae061-dcd8-4b15-bc34-4a95388d5f08 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:51.356809 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-8a0ae061-dcd8-4b15-bc34-4a95388d5f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:51.376454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:51.460163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 940/1879] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:48:51.461469 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d2a0e-95ef-4785-830c-ae393715189c None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:51.461802 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d2a0e-95ef-4785-830c-ae393715189c None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:51.462265 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d2a0e-95ef-4785-830c-ae393715189c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:51.503461 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-2a4d2a0e-95ef-4785-830c-ae393715189c None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:51.504743 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-2a4d2a0e-95ef-4785-830c-ae393715189c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:51.527382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 940/1880] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:48:51.528744 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ae061-dcd8-4b15-bc34-4a95388d5f08 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:51.544839 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ae061-dcd8-4b15-bc34-4a95388d5f08 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:51.558228 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5bedb-9c76-42b2-8a69-144c14fab57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:51.558927 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5bedb-9c76-42b2-8a69-144c14fab57f admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:51.559474 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5bedb-9c76-42b2-8a69-144c14fab57f admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:51.559761 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5bedb-9c76-42b2-8a69-144c14fab57f admin admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:51.560515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5bedb-9c76-42b2-8a69-144c14fab57f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:48:51.560862 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5bedb-9c76-42b2-8a69-144c14fab57f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:51.562571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5bedb-9c76-42b2-8a69-144c14fab57f admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:51.591674 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d2a0e-95ef-4785-830c-ae393715189c None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:51.592232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 941/1881] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:48:51.598177 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5bedb-9c76-42b2-8a69-144c14fab57f admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:51.598741 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-2a4d2a0e-95ef-4785-830c-ae393715189c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:51.609137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5bedb-9c76-42b2-8a69-144c14fab57f admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:51.614482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4480a9-be9f-421a-b93c-fb60c9848458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:51.620119 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4480a9-be9f-421a-b93c-fb60c9848458 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:51.620408 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4480a9-be9f-421a-b93c-fb60c9848458 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:51.620688 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4480a9-be9f-421a-b93c-fb60c9848458 admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:51.621410 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4480a9-be9f-421a-b93c-fb60c9848458 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:51.623428 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4480a9-be9f-421a-b93c-fb60c9848458 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:51.682328 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 942/1882] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:48:51 2022] POST /identity/v3/projects => generated 375 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:48:51.686568 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4480a9-be9f-421a-b93c-fb60c9848458 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:51.758464 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d2a0e-95ef-4785-830c-ae393715189c None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:51.782577 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4480a9-be9f-421a-b93c-fb60c9848458 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:51.788404 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66315e54-4aab-4201-9c54-cc15df60e38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:51.789343 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66315e54-4aab-4201-9c54-cc15df60e38f admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:51.789465 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66315e54-4aab-4201-9c54-cc15df60e38f admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:51.789645 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66315e54-4aab-4201-9c54-cc15df60e38f admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:51.790712 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66315e54-4aab-4201-9c54-cc15df60e38f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:51.793430 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66315e54-4aab-4201-9c54-cc15df60e38f admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:51.819344 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 943/1883] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:48:51 2022] POST /identity/v3/users => generated 288 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 14:48:51.822977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66315e54-4aab-4201-9c54-cc15df60e38f admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:51.835474 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66315e54-4aab-4201-9c54-cc15df60e38f admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:51.840921 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ede8f4-3e68-4dc1-8303-8b99fa500b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:51.841627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ede8f4-3e68-4dc1-8303-8b99fa500b61 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:51.841888 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ede8f4-3e68-4dc1-8303-8b99fa500b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:51.842241 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ede8f4-3e68-4dc1-8303-8b99fa500b61 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:51.842961 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ede8f4-3e68-4dc1-8303-8b99fa500b61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:51.843463 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ee6f3-8bee-4fbd-a90c-7a57abc7f284 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:51.844229 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ee6f3-8bee-4fbd-a90c-7a57abc7f284 service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:51.844575 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ee6f3-8bee-4fbd-a90c-7a57abc7f284 service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:51.844675 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ede8f4-3e68-4dc1-8303-8b99fa500b61 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:51.844918 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ee6f3-8bee-4fbd-a90c-7a57abc7f284 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:51.846790 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-105ee6f3-8bee-4fbd-a90c-7a57abc7f284 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:51.857388 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 944/1884] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 14:48:51.861282 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ede8f4-3e68-4dc1-8303-8b99fa500b61 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:51.861415 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ee6f3-8bee-4fbd-a90c-7a57abc7f284 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:51.862944 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ee6f3-8bee-4fbd-a90c-7a57abc7f284 service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:51.873980 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ede8f4-3e68-4dc1-8303-8b99fa500b61 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:51.877464 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 941/1885] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:48:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:48:51.897948 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c495bfd1-da64-43f0-8646-e6541beda722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:51.899289 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495bfd1-da64-43f0-8646-e6541beda722 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:51.899563 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495bfd1-da64-43f0-8646-e6541beda722 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:51.899854 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495bfd1-da64-43f0-8646-e6541beda722 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:51.900603 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c495bfd1-da64-43f0-8646-e6541beda722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:51.904480 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c495bfd1-da64-43f0-8646-e6541beda722 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:51.926555 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 945/1886] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:48:51.930538 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ee6f3-8bee-4fbd-a90c-7a57abc7f284 service nova] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:51.946306 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ee6f3-8bee-4fbd-a90c-7a57abc7f284 service nova] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:51.952448 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae1f2c-57e2-4de6-8aae-ef5bda442aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:51.953714 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae1f2c-57e2-4de6-8aae-ef5bda442aa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:51.954344 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae1f2c-57e2-4de6-8aae-ef5bda442aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:51.954847 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae1f2c-57e2-4de6-8aae-ef5bda442aa8 admin admin] PATH_INFO: `/v3/projects/035d61ce68154851993db7cc786512d4/users/50a3949b71224dbcbca9e2cc0875a33f/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:51.963907 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:51.988466 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eae1f2c-57e2-4de6-8aae-ef5bda442aa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=035d61ce68154851993db7cc786512d4, user_id=50a3949b71224dbcbca9e2cc0875a33f, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:51.995713 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eae1f2c-57e2-4de6-8aae-ef5bda442aa8 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:52.024774 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 942/1887] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:48:51 2022] PUT /identity/v3/projects/035d61ce68154851993db7cc786512d4/users/50a3949b71224dbcbca9e2cc0875a33f/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:48:52.036798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545cb2d5-e553-4348-a902-14dbbe290691 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:52.037489 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545cb2d5-e553-4348-a902-14dbbe290691 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:52.037919 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545cb2d5-e553-4348-a902-14dbbe290691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:52.081883 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-545cb2d5-e553-4348-a902-14dbbe290691 None None] MFA Rules not processed for user `50a3949b71224dbcbca9e2cc0875a33f`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:52.083310 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-545cb2d5-e553-4348-a902-14dbbe290691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:52.172601 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:52.253154 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 946/1888] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:52.272833 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae1f2c-57e2-4de6-8aae-ef5bda442aa8 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:52.274854 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-4eae1f2c-57e2-4de6-8aae-ef5bda442aa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:52.395706 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae1f2c-57e2-4de6-8aae-ef5bda442aa8 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:52.400799 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ccdc4a-da97-47a4-9d09-8f9899be0385 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:52.401634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccdc4a-da97-47a4-9d09-8f9899be0385 service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:52.402074 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccdc4a-da97-47a4-9d09-8f9899be0385 service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:52.402574 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccdc4a-da97-47a4-9d09-8f9899be0385 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:52.416510 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ccdc4a-da97-47a4-9d09-8f9899be0385 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:52.418969 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ccdc4a-da97-47a4-9d09-8f9899be0385 service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:52.445925 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 943/1889] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:48:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:48:53.918621 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa8365-b76e-44a8-99d0-84be9e23d415 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:53.919416 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa8365-b76e-44a8-99d0-84be9e23d415 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:53.919416 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa8365-b76e-44a8-99d0-84be9e23d415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:53.955346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-dbfa8365-b76e-44a8-99d0-84be9e23d415 None None] MFA Rules not processed for user `513b043962cf48c695c7c724bd08395f`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:48:53.956709 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-dbfa8365-b76e-44a8-99d0-84be9e23d415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:54.030571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 947/1890] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:48:53 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:48:54.044626 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ccdc4a-da97-47a4-9d09-8f9899be0385 service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:48:54.145377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ccdc4a-da97-47a4-9d09-8f9899be0385 service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:48:54.152249 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c829448-27d1-4a52-8868-1ee2cddb4601 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:48:54.153049 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c829448-27d1-4a52-8868-1ee2cddb4601 service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:48:54.153406 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c829448-27d1-4a52-8868-1ee2cddb4601 service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:48:54.153754 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c829448-27d1-4a52-8868-1ee2cddb4601 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:48:54.155921 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-8c829448-27d1-4a52-8868-1ee2cddb4601 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:48:54.235704 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c829448-27d1-4a52-8868-1ee2cddb4601 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:48:54.237342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c829448-27d1-4a52-8868-1ee2cddb4601 service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:48:54.258045 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 944/1891] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:48:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:49:03.614914 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa8365-b76e-44a8-99d0-84be9e23d415 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:03.616618 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-dbfa8365-b76e-44a8-99d0-84be9e23d415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:03.690862 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa8365-b76e-44a8-99d0-84be9e23d415 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:03.696548 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e20494a-65f8-485a-a51b-7086f9e72a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:03.697597 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20494a-65f8-485a-a51b-7086f9e72a94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:03.697900 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20494a-65f8-485a-a51b-7086f9e72a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:03.698237 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20494a-65f8-485a-a51b-7086f9e72a94 admin admin] PATH_INFO: `/v3/users/0432bf57115d453e88f561e29ea803f4` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:03.699682 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e20494a-65f8-485a-a51b-7086f9e72a94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0432bf57115d453e88f561e29ea803f4)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:03.701952 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e20494a-65f8-485a-a51b-7086f9e72a94 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:03.764744 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-6e20494a-65f8-485a-a51b-7086f9e72a94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0432bf57115d453e88f561e29ea803f4'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:03.785615 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-6e20494a-65f8-485a-a51b-7086f9e72a94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0432bf57115d453e88f561e29ea803f4'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:03.786696 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-6e20494a-65f8-485a-a51b-7086f9e72a94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0432bf57115d453e88f561e29ea803f4'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:03.787655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:03.788611 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-6e20494a-65f8-485a-a51b-7086f9e72a94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0432bf57115d453e88f561e29ea803f4'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:03.799253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-6e20494a-65f8-485a-a51b-7086f9e72a94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0432bf57115d453e88f561e29ea803f4'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:03.816171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 948/1892] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:49:03 2022] DELETE /identity/v3/users/0432bf57115d453e88f561e29ea803f4 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:49:03.820985 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5cbb46-ad57-4413-913f-404abd567e0a None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:03.821548 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5cbb46-ad57-4413-913f-404abd567e0a None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:03.821908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5cbb46-ad57-4413-913f-404abd567e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:03.840093 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-7d5cbb46-ad57-4413-913f-404abd567e0a None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:49:03.841412 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-7d5cbb46-ad57-4413-913f-404abd567e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:03.862567 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:03.920689 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 945/1893] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:49:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:49:03.930814 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e20494a-65f8-485a-a51b-7086f9e72a94 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:03.932495 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-6e20494a-65f8-485a-a51b-7086f9e72a94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:04.004610 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e20494a-65f8-485a-a51b-7086f9e72a94 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:04.010519 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc8bc4-91f2-4891-af09-7a95e83cbb0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:04.011269 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc8bc4-91f2-4891-af09-7a95e83cbb0f service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:04.011555 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc8bc4-91f2-4891-af09-7a95e83cbb0f service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:04.011828 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc8bc4-91f2-4891-af09-7a95e83cbb0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:04.022160 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc8bc4-91f2-4891-af09-7a95e83cbb0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:04.023666 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc8bc4-91f2-4891-af09-7a95e83cbb0f service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:04.034335 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 949/1894] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:49:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 14:49:04.359446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5cbb46-ad57-4413-913f-404abd567e0a None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:04.361083 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-7d5cbb46-ad57-4413-913f-404abd567e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:04.375372 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5cbb46-ad57-4413-913f-404abd567e0a None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:04.380428 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef73fd-949e-481d-833e-e266c07bc0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:04.381458 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef73fd-949e-481d-833e-e266c07bc0e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:04.381945 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef73fd-949e-481d-833e-e266c07bc0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:04.382302 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef73fd-949e-481d-833e-e266c07bc0e0 admin admin] PATH_INFO: `/v3/projects/dfe7c5ee1264469b88c7c4f43e7ef048` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:04.383695 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef73fd-949e-481d-833e-e266c07bc0e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfe7c5ee1264469b88c7c4f43e7ef048)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:04.385749 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef73fd-949e-481d-833e-e266c07bc0e0 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:04.443715 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-aeef73fd-949e-481d-833e-e266c07bc0e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfe7c5ee1264469b88c7c4f43e7ef048', 'request_id': 'req-aeef73fd-949e-481d-833e-e266c07bc0e0'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:04.449638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-aeef73fd-949e-481d-833e-e266c07bc0e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfe7c5ee1264469b88c7c4f43e7ef048', 'request_id': 'req-aeef73fd-949e-481d-833e-e266c07bc0e0'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:04.456638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-aeef73fd-949e-481d-833e-e266c07bc0e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfe7c5ee1264469b88c7c4f43e7ef048', 'request_id': 'req-aeef73fd-949e-481d-833e-e266c07bc0e0'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:04.475472 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-aeef73fd-949e-481d-833e-e266c07bc0e0 admin admin] The token cache is being invalidate because project dfe7c5ee1264469b88c7c4f43e7ef048 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:49:04.475765 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-aeef73fd-949e-481d-833e-e266c07bc0e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:04.476594 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-aeef73fd-949e-481d-833e-e266c07bc0e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:04.478304 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 946/1895] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:49:04 2022] DELETE /identity/v3/projects/dfe7c5ee1264469b88c7c4f43e7ef048 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:49:04.483003 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc8bc4-91f2-4891-af09-7a95e83cbb0f service neutron] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:04.495895 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc8bc4-91f2-4891-af09-7a95e83cbb0f service neutron] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:04.583183 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb708fa1-76fc-4b00-987a-347f67e639ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:04.584161 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb708fa1-76fc-4b00-987a-347f67e639ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:04.584573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb708fa1-76fc-4b00-987a-347f67e639ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:04.584866 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb708fa1-76fc-4b00-987a-347f67e639ab admin admin] PATH_INFO: `/v3/users/1729d01bb2284886852622a8b40f3424` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:04.586576 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb708fa1-76fc-4b00-987a-347f67e639ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=1729d01bb2284886852622a8b40f3424)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:04.588586 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb708fa1-76fc-4b00-987a-347f67e639ab admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:04.642885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-cb708fa1-76fc-4b00-987a-347f67e639ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1729d01bb2284886852622a8b40f3424'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:04.660153 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-cb708fa1-76fc-4b00-987a-347f67e639ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1729d01bb2284886852622a8b40f3424'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:04.660989 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-cb708fa1-76fc-4b00-987a-347f67e639ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1729d01bb2284886852622a8b40f3424'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:04.661682 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-cb708fa1-76fc-4b00-987a-347f67e639ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1729d01bb2284886852622a8b40f3424'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:04.670112 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-cb708fa1-76fc-4b00-987a-347f67e639ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1729d01bb2284886852622a8b40f3424'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:04.688425 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 950/1896] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:49:04 2022] DELETE /identity/v3/users/1729d01bb2284886852622a8b40f3424 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 14:49:04.982536 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef73fd-949e-481d-833e-e266c07bc0e0 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:04.984114 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-aeef73fd-949e-481d-833e-e266c07bc0e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:05.024819 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef73fd-949e-481d-833e-e266c07bc0e0 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:05.109409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b636b-4d22-47ec-bc84-f1292006f8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:05.110420 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b636b-4d22-47ec-bc84-f1292006f8bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:05.110782 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b636b-4d22-47ec-bc84-f1292006f8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:05.111196 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b636b-4d22-47ec-bc84-f1292006f8bf admin admin] PATH_INFO: `/v3/projects/a19d5bde23624fbd9daab52c3c2c3d08` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:05.112463 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b636b-4d22-47ec-bc84-f1292006f8bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=a19d5bde23624fbd9daab52c3c2c3d08)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:05.114482 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b636b-4d22-47ec-bc84-f1292006f8bf admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:05.169940 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ba5b636b-4d22-47ec-bc84-f1292006f8bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a19d5bde23624fbd9daab52c3c2c3d08', 'request_id': 'req-ba5b636b-4d22-47ec-bc84-f1292006f8bf'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:05.171435 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:05.177913 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ba5b636b-4d22-47ec-bc84-f1292006f8bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a19d5bde23624fbd9daab52c3c2c3d08', 'request_id': 'req-ba5b636b-4d22-47ec-bc84-f1292006f8bf'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:05.185256 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ba5b636b-4d22-47ec-bc84-f1292006f8bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a19d5bde23624fbd9daab52c3c2c3d08', 'request_id': 'req-ba5b636b-4d22-47ec-bc84-f1292006f8bf'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:05.199723 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ba5b636b-4d22-47ec-bc84-f1292006f8bf admin admin] The token cache is being invalidate because project a19d5bde23624fbd9daab52c3c2c3d08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:49:05.200033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ba5b636b-4d22-47ec-bc84-f1292006f8bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:05.203505 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ba5b636b-4d22-47ec-bc84-f1292006f8bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:05.207742 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 947/1897] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:49:04 2022] DELETE /identity/v3/projects/a19d5bde23624fbd9daab52c3c2c3d08 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:49:05.210513 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8f2fd-66d3-4bfe-b7e9-decd1683f862 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:05.210810 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8f2fd-66d3-4bfe-b7e9-decd1683f862 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:05.211135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8f2fd-66d3-4bfe-b7e9-decd1683f862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:05.217768 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:05.233105 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-c9d8f2fd-66d3-4bfe-b7e9-decd1683f862 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:49:05.234414 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-c9d8f2fd-66d3-4bfe-b7e9-decd1683f862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:05.305289 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 951/1898] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:49:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:49:05.315877 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617da971-1f3f-49bd-afe8-1405712b1df0 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:05.316151 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617da971-1f3f-49bd-afe8-1405712b1df0 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:05.316440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617da971-1f3f-49bd-afe8-1405712b1df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:05.339496 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-617da971-1f3f-49bd-afe8-1405712b1df0 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:49:05.340739 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-617da971-1f3f-49bd-afe8-1405712b1df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:05.349198 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 948/1899] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:49:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:49:05.353701 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8f2fd-66d3-4bfe-b7e9-decd1683f862 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:05.363716 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8f2fd-66d3-4bfe-b7e9-decd1683f862 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:05.368780 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5232413-010e-431b-b1e1-05c88f59b3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:05.369446 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5232413-010e-431b-b1e1-05c88f59b3d8 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:05.369733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5232413-010e-431b-b1e1-05c88f59b3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:05.370006 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5232413-010e-431b-b1e1-05c88f59b3d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:05.370733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5232413-010e-431b-b1e1-05c88f59b3d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:49:05.371037 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5232413-010e-431b-b1e1-05c88f59b3d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:05.372695 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5232413-010e-431b-b1e1-05c88f59b3d8 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:05.385788 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 952/1900] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:49:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:49:05.390740 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617da971-1f3f-49bd-afe8-1405712b1df0 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:05.401042 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617da971-1f3f-49bd-afe8-1405712b1df0 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:05.407579 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d681bb34-e27b-46a7-8dcb-7f11ebcc2759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:05.408589 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681bb34-e27b-46a7-8dcb-7f11ebcc2759 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:05.409074 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681bb34-e27b-46a7-8dcb-7f11ebcc2759 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:05.409572 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681bb34-e27b-46a7-8dcb-7f11ebcc2759 admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:05.410937 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d681bb34-e27b-46a7-8dcb-7f11ebcc2759 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:05.414560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d681bb34-e27b-46a7-8dcb-7f11ebcc2759 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:05.450555 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 949/1901] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:49:05 2022] POST /identity/v3/projects => generated 357 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:49:05.455657 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5232413-010e-431b-b1e1-05c88f59b3d8 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:05.466498 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5232413-010e-431b-b1e1-05c88f59b3d8 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:05.533492 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252dc85-317f-47a1-b514-bda4ac66bd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:05.534282 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252dc85-317f-47a1-b514-bda4ac66bd4e admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:05.534599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252dc85-317f-47a1-b514-bda4ac66bd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:05.534909 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252dc85-317f-47a1-b514-bda4ac66bd4e admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:05.535765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7252dc85-317f-47a1-b514-bda4ac66bd4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:05.538078 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7252dc85-317f-47a1-b514-bda4ac66bd4e admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:05.564642 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 953/1902] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:49:05 2022] POST /identity/v3/users => generated 279 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:49:05.569494 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d681bb34-e27b-46a7-8dcb-7f11ebcc2759 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:05.580625 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d681bb34-e27b-46a7-8dcb-7f11ebcc2759 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:05.585854 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9428d0e0-6e76-4fea-8d8f-adc6e233b9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:05.586512 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428d0e0-6e76-4fea-8d8f-adc6e233b9da admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:05.586808 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428d0e0-6e76-4fea-8d8f-adc6e233b9da admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:05.587108 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428d0e0-6e76-4fea-8d8f-adc6e233b9da admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:05.587861 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9428d0e0-6e76-4fea-8d8f-adc6e233b9da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:05.589540 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9428d0e0-6e76-4fea-8d8f-adc6e233b9da admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:05.602058 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 950/1903] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:49:05.606542 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252dc85-317f-47a1-b514-bda4ac66bd4e admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:05.618093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252dc85-317f-47a1-b514-bda4ac66bd4e admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:05.625264 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfcd18c-b894-4fbc-a7f3-9931ae2b9df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:05.626225 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfcd18c-b894-4fbc-a7f3-9931ae2b9df8 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:05.626519 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfcd18c-b894-4fbc-a7f3-9931ae2b9df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:05.626795 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfcd18c-b894-4fbc-a7f3-9931ae2b9df8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:05.627896 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfcd18c-b894-4fbc-a7f3-9931ae2b9df8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:05.629552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfcd18c-b894-4fbc-a7f3-9931ae2b9df8 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:05.649932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 954/1904] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:49:05.650622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9428d0e0-6e76-4fea-8d8f-adc6e233b9da admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:05.666340 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9428d0e0-6e76-4fea-8d8f-adc6e233b9da admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:05.672686 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5012edb-d5c5-43a9-a7eb-301c2a27ad75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:05.673388 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5012edb-d5c5-43a9-a7eb-301c2a27ad75 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:05.673716 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5012edb-d5c5-43a9-a7eb-301c2a27ad75 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:05.674628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5012edb-d5c5-43a9-a7eb-301c2a27ad75 admin admin] PATH_INFO: `/v3/projects/9e43b0850f04480ebc5c13622f97b1ea/users/fa0abe709b7449b4953426f985729100/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:05.698692 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5012edb-d5c5-43a9-a7eb-301c2a27ad75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e43b0850f04480ebc5c13622f97b1ea, user_id=fa0abe709b7449b4953426f985729100, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:05.701516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5012edb-d5c5-43a9-a7eb-301c2a27ad75 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:05.717729 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 951/1905] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:49:05 2022] PUT /identity/v3/projects/9e43b0850f04480ebc5c13622f97b1ea/users/fa0abe709b7449b4953426f985729100/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:49:05.729379 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7789a5b-fb58-4b23-a494-f0b5fdb86eb3 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:05.729782 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7789a5b-fb58-4b23-a494-f0b5fdb86eb3 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:05.730186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7789a5b-fb58-4b23-a494-f0b5fdb86eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:05.761586 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-e7789a5b-fb58-4b23-a494-f0b5fdb86eb3 None None] MFA Rules not processed for user `fa0abe709b7449b4953426f985729100`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:49:05.763095 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-e7789a5b-fb58-4b23-a494-f0b5fdb86eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:05.853655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:05.867240 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 955/1906] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:49:05 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:49:05.870671 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5012edb-d5c5-43a9-a7eb-301c2a27ad75 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:05.949560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5012edb-d5c5-43a9-a7eb-301c2a27ad75 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:05.954577 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99376b-6668-484a-8488-b5d6bc4c63a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:05.955369 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99376b-6668-484a-8488-b5d6bc4c63a9 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:05.955659 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99376b-6668-484a-8488-b5d6bc4c63a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:05.955944 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99376b-6668-484a-8488-b5d6bc4c63a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:05.956734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99376b-6668-484a-8488-b5d6bc4c63a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:05.958807 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99376b-6668-484a-8488-b5d6bc4c63a9 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:05.986576 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 952/1907] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:49:05 2022] POST /identity/v3/projects => generated 357 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:49:05.990920 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7789a5b-fb58-4b23-a494-f0b5fdb86eb3 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:06.002502 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7789a5b-fb58-4b23-a494-f0b5fdb86eb3 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:06.069698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64daace-0a76-4770-b851-6b0c4fe409d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:06.070436 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64daace-0a76-4770-b851-6b0c4fe409d5 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:06.070725 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64daace-0a76-4770-b851-6b0c4fe409d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:06.071013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64daace-0a76-4770-b851-6b0c4fe409d5 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:06.071956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64daace-0a76-4770-b851-6b0c4fe409d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:06.074089 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64daace-0a76-4770-b851-6b0c4fe409d5 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:06.097117 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 956/1908] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:49:05 2022] POST /identity/v3/users => generated 285 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:49:06.101788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99376b-6668-484a-8488-b5d6bc4c63a9 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:06.114556 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99376b-6668-484a-8488-b5d6bc4c63a9 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:06.121946 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdee3a3-e925-4b6c-9b3d-0b8de212d749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:06.122976 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdee3a3-e925-4b6c-9b3d-0b8de212d749 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:06.123294 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdee3a3-e925-4b6c-9b3d-0b8de212d749 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:06.123747 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdee3a3-e925-4b6c-9b3d-0b8de212d749 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:06.125236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdee3a3-e925-4b6c-9b3d-0b8de212d749 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:06.126992 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdee3a3-e925-4b6c-9b3d-0b8de212d749 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:06.139936 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 953/1909] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:49:06.144825 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64daace-0a76-4770-b851-6b0c4fe409d5 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:06.155748 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64daace-0a76-4770-b851-6b0c4fe409d5 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:06.161162 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4a6f3-f5d6-4ce2-a8e6-8c179396f2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:06.162069 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4a6f3-f5d6-4ce2-a8e6-8c179396f2a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:06.162349 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4a6f3-f5d6-4ce2-a8e6-8c179396f2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:06.162627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4a6f3-f5d6-4ce2-a8e6-8c179396f2a1 admin admin] PATH_INFO: `/v3/projects/aadf14ba691d4ea483f095b19bbe734d/users/1f6d963ff4124a7ea8042b175143474b/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:06.178243 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b4a6f3-f5d6-4ce2-a8e6-8c179396f2a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aadf14ba691d4ea483f095b19bbe734d, user_id=1f6d963ff4124a7ea8042b175143474b, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:06.180991 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b4a6f3-f5d6-4ce2-a8e6-8c179396f2a1 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:06.196602 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 957/1910] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:49:06 2022] PUT /identity/v3/projects/aadf14ba691d4ea483f095b19bbe734d/users/1f6d963ff4124a7ea8042b175143474b/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:49:06.200979 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdee3a3-e925-4b6c-9b3d-0b8de212d749 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:06.212722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdee3a3-e925-4b6c-9b3d-0b8de212d749 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:06.277641 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:06.293125 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f031b755-d77a-4cf1-8b02-0a5d7e6ea9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:06.293828 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031b755-d77a-4cf1-8b02-0a5d7e6ea9a9 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:06.294117 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031b755-d77a-4cf1-8b02-0a5d7e6ea9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:06.294392 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031b755-d77a-4cf1-8b02-0a5d7e6ea9a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:06.295201 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f031b755-d77a-4cf1-8b02-0a5d7e6ea9a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:06.296958 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f031b755-d77a-4cf1-8b02-0a5d7e6ea9a9 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:06.308316 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 954/1911] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:49:06.312905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4a6f3-f5d6-4ce2-a8e6-8c179396f2a1 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:06.323703 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4a6f3-f5d6-4ce2-a8e6-8c179396f2a1 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:06.328795 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76984241-9699-40a0-ac28-6a45e7560772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:06.329723 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76984241-9699-40a0-ac28-6a45e7560772 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:06.330008 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76984241-9699-40a0-ac28-6a45e7560772 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:06.330287 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76984241-9699-40a0-ac28-6a45e7560772 admin admin] PATH_INFO: `/v3/projects/aadf14ba691d4ea483f095b19bbe734d/users/1f6d963ff4124a7ea8042b175143474b/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:06.332387 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76984241-9699-40a0-ac28-6a45e7560772 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aadf14ba691d4ea483f095b19bbe734d, user_id=1f6d963ff4124a7ea8042b175143474b, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:06.335065 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76984241-9699-40a0-ac28-6a45e7560772 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:06.349763 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 958/1912] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:49:06 2022] PUT /identity/v3/projects/aadf14ba691d4ea483f095b19bbe734d/users/1f6d963ff4124a7ea8042b175143474b/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:49:06.362584 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7005426-cc02-495c-b0a6-51060b1f0058 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:06.362877 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7005426-cc02-495c-b0a6-51060b1f0058 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:06.363258 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7005426-cc02-495c-b0a6-51060b1f0058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:06.393241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-c7005426-cc02-495c-b0a6-51060b1f0058 None None] MFA Rules not processed for user `1f6d963ff4124a7ea8042b175143474b`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:49:06.394490 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-c7005426-cc02-495c-b0a6-51060b1f0058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:06.475104 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 955/1913] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:49:06 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:49:06.487038 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76984241-9699-40a0-ac28-6a45e7560772 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:06.488806 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-76984241-9699-40a0-ac28-6a45e7560772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:06.554135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76984241-9699-40a0-ac28-6a45e7560772 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:06.558981 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a52f5f-ea52-4901-9bbd-0e93cf439805 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:06.559717 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a52f5f-ea52-4901-9bbd-0e93cf439805 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:06.559988 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a52f5f-ea52-4901-9bbd-0e93cf439805 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:06.560289 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a52f5f-ea52-4901-9bbd-0e93cf439805 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:06.573968 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a52f5f-ea52-4901-9bbd-0e93cf439805 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:06.576370 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a52f5f-ea52-4901-9bbd-0e93cf439805 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:06.587804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 959/1914] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:49:06 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:49:06.670544 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7005426-cc02-495c-b0a6-51060b1f0058 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:06.681618 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7005426-cc02-495c-b0a6-51060b1f0058 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:06.686076 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef2c7d7-aab6-4924-89d4-20934fb4618c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:06.686797 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2c7d7-aab6-4924-89d4-20934fb4618c service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:06.687082 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2c7d7-aab6-4924-89d4-20934fb4618c service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:06.687531 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2c7d7-aab6-4924-89d4-20934fb4618c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:06.754429 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef2c7d7-aab6-4924-89d4-20934fb4618c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:06.755930 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef2c7d7-aab6-4924-89d4-20934fb4618c service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:06.766472 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 956/1915] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:49:06 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:49:16.583034 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=71,keystone:INSERT=5 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:16.763223 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=69 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:25.042459 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ef848-441a-45bc-83d3-ea179d013263 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:25.042939 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ef848-441a-45bc-83d3-ea179d013263 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:25.043080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ef848-441a-45bc-83d3-ea179d013263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:25.060476 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-f89ef848-441a-45bc-83d3-ea179d013263 None None] MFA Rules not processed for user `dd610b017e7d4bd786db6bcd08e60722`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:49:25.061610 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-f89ef848-441a-45bc-83d3-ea179d013263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:25.119813 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 960/1916] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:49:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:49:26.561765 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef2c7d7-aab6-4924-89d4-20934fb4618c service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:26.563272 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-bef2c7d7-aab6-4924-89d4-20934fb4618c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:26.635773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef2c7d7-aab6-4924-89d4-20934fb4618c service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:26.640410 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df43344d-f55e-48ed-979d-15d6d86e457b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:26.641370 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43344d-f55e-48ed-979d-15d6d86e457b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:26.641677 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43344d-f55e-48ed-979d-15d6d86e457b admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:26.641952 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43344d-f55e-48ed-979d-15d6d86e457b admin admin] PATH_INFO: `/v3/users/50a3949b71224dbcbca9e2cc0875a33f` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:26.643212 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df43344d-f55e-48ed-979d-15d6d86e457b admin admin] RBAC: Authorizing `identity:delete_user(user_id=50a3949b71224dbcbca9e2cc0875a33f)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:26.645143 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df43344d-f55e-48ed-979d-15d6d86e457b admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:26.702230 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-df43344d-f55e-48ed-979d-15d6d86e457b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50a3949b71224dbcbca9e2cc0875a33f'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:26.716068 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-df43344d-f55e-48ed-979d-15d6d86e457b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50a3949b71224dbcbca9e2cc0875a33f'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:26.717073 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-df43344d-f55e-48ed-979d-15d6d86e457b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50a3949b71224dbcbca9e2cc0875a33f'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:26.718677 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-df43344d-f55e-48ed-979d-15d6d86e457b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50a3949b71224dbcbca9e2cc0875a33f'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:26.725982 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-df43344d-f55e-48ed-979d-15d6d86e457b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50a3949b71224dbcbca9e2cc0875a33f'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:26.742362 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 957/1917] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:49:26 2022] DELETE /identity/v3/users/50a3949b71224dbcbca9e2cc0875a33f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:49:26.746940 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80096a-701c-45fd-9f19-fea51b536da7 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:26.747458 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80096a-701c-45fd-9f19-fea51b536da7 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:26.747570 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80096a-701c-45fd-9f19-fea51b536da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:26.768732 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-fa80096a-701c-45fd-9f19-fea51b536da7 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:49:26.769966 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-fa80096a-701c-45fd-9f19-fea51b536da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:26.840358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 961/1918] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:49:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:49:26.850258 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df43344d-f55e-48ed-979d-15d6d86e457b admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:26.851735 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-df43344d-f55e-48ed-979d-15d6d86e457b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:26.934004 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df43344d-f55e-48ed-979d-15d6d86e457b admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:26.938596 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917fb056-f9f8-43dc-ad37-64fb2e6e1ea5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:26.939473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917fb056-f9f8-43dc-ad37-64fb2e6e1ea5 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:26.939800 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917fb056-f9f8-43dc-ad37-64fb2e6e1ea5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:26.940214 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917fb056-f9f8-43dc-ad37-64fb2e6e1ea5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:26.952722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917fb056-f9f8-43dc-ad37-64fb2e6e1ea5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:26.954237 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917fb056-f9f8-43dc-ad37-64fb2e6e1ea5 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:26.965073 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 958/1919] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:49:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:49:27.357645 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80096a-701c-45fd-9f19-fea51b536da7 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:27.359205 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-fa80096a-701c-45fd-9f19-fea51b536da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:27.377430 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80096a-701c-45fd-9f19-fea51b536da7 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:27.386745 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e6e0a9-6d9d-46a5-a259-f707a4681249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:27.388116 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e6e0a9-6d9d-46a5-a259-f707a4681249 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:27.388703 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e6e0a9-6d9d-46a5-a259-f707a4681249 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:27.389079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e6e0a9-6d9d-46a5-a259-f707a4681249 admin admin] PATH_INFO: `/v3/projects/035d61ce68154851993db7cc786512d4` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:27.393099 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e6e0a9-6d9d-46a5-a259-f707a4681249 admin admin] RBAC: Authorizing `identity:delete_project(project_id=035d61ce68154851993db7cc786512d4)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:27.396858 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e6e0a9-6d9d-46a5-a259-f707a4681249 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:27.474628 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-53e6e0a9-6d9d-46a5-a259-f707a4681249 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '035d61ce68154851993db7cc786512d4', 'request_id': 'req-53e6e0a9-6d9d-46a5-a259-f707a4681249'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:27.478704 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-53e6e0a9-6d9d-46a5-a259-f707a4681249 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '035d61ce68154851993db7cc786512d4', 'request_id': 'req-53e6e0a9-6d9d-46a5-a259-f707a4681249'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:27.492415 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-53e6e0a9-6d9d-46a5-a259-f707a4681249 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '035d61ce68154851993db7cc786512d4', 'request_id': 'req-53e6e0a9-6d9d-46a5-a259-f707a4681249'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:27.508541 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-53e6e0a9-6d9d-46a5-a259-f707a4681249 admin admin] The token cache is being invalidate because project 035d61ce68154851993db7cc786512d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:49:27.509008 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-53e6e0a9-6d9d-46a5-a259-f707a4681249 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:27.509804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-53e6e0a9-6d9d-46a5-a259-f707a4681249 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:27.511976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 962/1920] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:49:27 2022] DELETE /identity/v3/projects/035d61ce68154851993db7cc786512d4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:49:27.516905 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9dcab8-0e94-461c-bbb1-abc00c6fd5c0 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:27.517541 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9dcab8-0e94-461c-bbb1-abc00c6fd5c0 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:27.517993 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9dcab8-0e94-461c-bbb1-abc00c6fd5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:27.553334 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-1b9dcab8-0e94-461c-bbb1-abc00c6fd5c0 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:49:27.556399 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-1b9dcab8-0e94-461c-bbb1-abc00c6fd5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:27.584742 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:27.641047 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 959/1921] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:49:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:49:27.648437 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51598acf-1c9c-40ca-9457-ad20d7cab3e4 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:27.648766 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51598acf-1c9c-40ca-9457-ad20d7cab3e4 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:27.649451 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51598acf-1c9c-40ca-9457-ad20d7cab3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:27.678692 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-51598acf-1c9c-40ca-9457-ad20d7cab3e4 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:49:27.680354 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-51598acf-1c9c-40ca-9457-ad20d7cab3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:27.690826 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 963/1922] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:49:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:49:27.695135 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9dcab8-0e94-461c-bbb1-abc00c6fd5c0 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:27.705765 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9dcab8-0e94-461c-bbb1-abc00c6fd5c0 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:27.710875 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46344f-4615-4013-ae22-19def1a7e75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:27.711695 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46344f-4615-4013-ae22-19def1a7e75f admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:27.711978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46344f-4615-4013-ae22-19def1a7e75f admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:27.712257 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46344f-4615-4013-ae22-19def1a7e75f admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:27.713006 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db46344f-4615-4013-ae22-19def1a7e75f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:49:27.713312 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db46344f-4615-4013-ae22-19def1a7e75f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:27.715001 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db46344f-4615-4013-ae22-19def1a7e75f admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:27.737894 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 960/1923] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:49:27.743191 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51598acf-1c9c-40ca-9457-ad20d7cab3e4 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:27.754666 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51598acf-1c9c-40ca-9457-ad20d7cab3e4 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:27.759719 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d716f-c791-4bb1-a591-9781583338d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:27.760724 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d716f-c791-4bb1-a591-9781583338d2 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:27.761176 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d716f-c791-4bb1-a591-9781583338d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:27.761545 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d716f-c791-4bb1-a591-9781583338d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:27.762310 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d716f-c791-4bb1-a591-9781583338d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:27.764640 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d716f-c791-4bb1-a591-9781583338d2 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:27.792215 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:27.804607 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46344f-4615-4013-ae22-19def1a7e75f admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:27.805087 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 964/1924] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:49:27 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:49:27.817309 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46344f-4615-4013-ae22-19def1a7e75f admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:27.914182 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d038690-53d6-455b-9f9a-bb53223b043c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:27.914951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d038690-53d6-455b-9f9a-bb53223b043c admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:27.915271 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d038690-53d6-455b-9f9a-bb53223b043c admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:27.915550 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d038690-53d6-455b-9f9a-bb53223b043c admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:27.916346 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d038690-53d6-455b-9f9a-bb53223b043c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:27.918311 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d038690-53d6-455b-9f9a-bb53223b043c admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:27.951719 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 961/1925] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:49:27 2022] POST /identity/v3/users => generated 288 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 14:49:27.955788 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d716f-c791-4bb1-a591-9781583338d2 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:27.986262 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d716f-c791-4bb1-a591-9781583338d2 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:28.020232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751cc7e8-d88a-42f4-abbd-16e956c8292a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:28.020952 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751cc7e8-d88a-42f4-abbd-16e956c8292a admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:28.021249 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751cc7e8-d88a-42f4-abbd-16e956c8292a admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:28.021763 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751cc7e8-d88a-42f4-abbd-16e956c8292a admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:28.022682 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751cc7e8-d88a-42f4-abbd-16e956c8292a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:28.025703 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751cc7e8-d88a-42f4-abbd-16e956c8292a admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:28.046988 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 965/1926] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:49:28.051345 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d038690-53d6-455b-9f9a-bb53223b043c admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:28.062786 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d038690-53d6-455b-9f9a-bb53223b043c admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:28.068302 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0e921-943b-41da-ba49-090b49d3af38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:28.068993 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0e921-943b-41da-ba49-090b49d3af38 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:28.069594 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0e921-943b-41da-ba49-090b49d3af38 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:28.069885 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0e921-943b-41da-ba49-090b49d3af38 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:28.070666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee0e921-943b-41da-ba49-090b49d3af38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:28.072361 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee0e921-943b-41da-ba49-090b49d3af38 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:28.084908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 962/1927] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 14:49:28.091305 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751cc7e8-d88a-42f4-abbd-16e956c8292a admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:28.110436 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751cc7e8-d88a-42f4-abbd-16e956c8292a admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:28.117200 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d182bbc-597e-4394-878a-3be04c6ffdf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:28.118165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d182bbc-597e-4394-878a-3be04c6ffdf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:28.118453 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d182bbc-597e-4394-878a-3be04c6ffdf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:28.118769 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d182bbc-597e-4394-878a-3be04c6ffdf3 admin admin] PATH_INFO: `/v3/projects/29bdc06e08b34c7aaf8d8dffc296eb07/users/6631ef3f6fd44e1083e54d76c0184301/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:28.135394 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d182bbc-597e-4394-878a-3be04c6ffdf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29bdc06e08b34c7aaf8d8dffc296eb07, user_id=6631ef3f6fd44e1083e54d76c0184301, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:28.138193 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d182bbc-597e-4394-878a-3be04c6ffdf3 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:28.153439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 966/1928] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:49:28 2022] PUT /identity/v3/projects/29bdc06e08b34c7aaf8d8dffc296eb07/users/6631ef3f6fd44e1083e54d76c0184301/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:49:28.165790 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba91381-7591-4f2a-9524-130647cf74c4 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:28.166593 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba91381-7591-4f2a-9524-130647cf74c4 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:28.167038 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba91381-7591-4f2a-9524-130647cf74c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:28.198482 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-2ba91381-7591-4f2a-9524-130647cf74c4 None None] MFA Rules not processed for user `6631ef3f6fd44e1083e54d76c0184301`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:49:28.199976 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-2ba91381-7591-4f2a-9524-130647cf74c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:28.284730 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 963/1929] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:49:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:49:28.294414 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d182bbc-597e-4394-878a-3be04c6ffdf3 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:28.296241 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-1d182bbc-597e-4394-878a-3be04c6ffdf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:28.363961 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d182bbc-597e-4394-878a-3be04c6ffdf3 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:28.367935 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5daffb-f8bd-4173-94ed-298db11a0e2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:28.368609 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5daffb-f8bd-4173-94ed-298db11a0e2a service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:28.368897 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5daffb-f8bd-4173-94ed-298db11a0e2a service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:28.369172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5daffb-f8bd-4173-94ed-298db11a0e2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:28.378510 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5daffb-f8bd-4173-94ed-298db11a0e2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:28.380004 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5daffb-f8bd-4173-94ed-298db11a0e2a service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:28.392941 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 967/1930] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:49:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:49:29.847193 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba91381-7591-4f2a-9524-130647cf74c4 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:29.847193 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-2ba91381-7591-4f2a-9524-130647cf74c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:29.903469 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:29.981744 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba91381-7591-4f2a-9524-130647cf74c4 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:29.987250 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca1e84-1513-440c-925a-7f79b4957efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:29.988221 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca1e84-1513-440c-925a-7f79b4957efa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:29.988515 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca1e84-1513-440c-925a-7f79b4957efa admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:29.988799 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca1e84-1513-440c-925a-7f79b4957efa admin admin] PATH_INFO: `/v3/users/fa0abe709b7449b4953426f985729100` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:29.999919 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca1e84-1513-440c-925a-7f79b4957efa admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa0abe709b7449b4953426f985729100)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:30.002540 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca1e84-1513-440c-925a-7f79b4957efa admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:30.080166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-afca1e84-1513-440c-925a-7f79b4957efa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa0abe709b7449b4953426f985729100'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:30.101379 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-afca1e84-1513-440c-925a-7f79b4957efa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa0abe709b7449b4953426f985729100'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:30.102480 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-afca1e84-1513-440c-925a-7f79b4957efa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa0abe709b7449b4953426f985729100'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:30.103556 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-afca1e84-1513-440c-925a-7f79b4957efa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa0abe709b7449b4953426f985729100'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:30.112194 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-afca1e84-1513-440c-925a-7f79b4957efa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa0abe709b7449b4953426f985729100'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:30.136253 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 964/1931] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:49:29 2022] DELETE /identity/v3/users/fa0abe709b7449b4953426f985729100 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:49:30.141762 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c421d-9312-4d4a-90fa-727514dcb550 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:30.142267 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c421d-9312-4d4a-90fa-727514dcb550 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:30.142434 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c421d-9312-4d4a-90fa-727514dcb550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:30.164576 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-a08c421d-9312-4d4a-90fa-727514dcb550 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:49:30.166006 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-a08c421d-9312-4d4a-90fa-727514dcb550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:30.242749 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 968/1932] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:49:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:49:30.254256 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca1e84-1513-440c-925a-7f79b4957efa admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:30.254474 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-afca1e84-1513-440c-925a-7f79b4957efa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:30.337226 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca1e84-1513-440c-925a-7f79b4957efa admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:30.347294 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def1ed2-7393-4dbf-92c9-8b1d3b9f7d9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:30.348327 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def1ed2-7393-4dbf-92c9-8b1d3b9f7d9b service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:30.348797 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def1ed2-7393-4dbf-92c9-8b1d3b9f7d9b service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:30.349249 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def1ed2-7393-4dbf-92c9-8b1d3b9f7d9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:30.363290 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6def1ed2-7393-4dbf-92c9-8b1d3b9f7d9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:30.364710 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6def1ed2-7393-4dbf-92c9-8b1d3b9f7d9b service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:30.379423 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 965/1933] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:49:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:49:30.698855 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c421d-9312-4d4a-90fa-727514dcb550 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:30.700461 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-a08c421d-9312-4d4a-90fa-727514dcb550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:30.717915 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c421d-9312-4d4a-90fa-727514dcb550 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:30.724069 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cfdc39-8235-4944-ad21-4550c2f345d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:30.725106 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cfdc39-8235-4944-ad21-4550c2f345d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:30.725584 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cfdc39-8235-4944-ad21-4550c2f345d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:30.725898 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cfdc39-8235-4944-ad21-4550c2f345d5 admin admin] PATH_INFO: `/v3/projects/9e43b0850f04480ebc5c13622f97b1ea` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:30.728556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cfdc39-8235-4944-ad21-4550c2f345d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e43b0850f04480ebc5c13622f97b1ea)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:30.730916 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cfdc39-8235-4944-ad21-4550c2f345d5 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:30.814996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-40cfdc39-8235-4944-ad21-4550c2f345d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e43b0850f04480ebc5c13622f97b1ea', 'request_id': 'req-40cfdc39-8235-4944-ad21-4550c2f345d5'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:30.823238 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-40cfdc39-8235-4944-ad21-4550c2f345d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e43b0850f04480ebc5c13622f97b1ea', 'request_id': 'req-40cfdc39-8235-4944-ad21-4550c2f345d5'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:30.837642 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:30.839393 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-40cfdc39-8235-4944-ad21-4550c2f345d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e43b0850f04480ebc5c13622f97b1ea', 'request_id': 'req-40cfdc39-8235-4944-ad21-4550c2f345d5'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:30.855292 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-40cfdc39-8235-4944-ad21-4550c2f345d5 admin admin] The token cache is being invalidate because project 9e43b0850f04480ebc5c13622f97b1ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:49:30.855585 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-40cfdc39-8235-4944-ad21-4550c2f345d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:30.856242 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-40cfdc39-8235-4944-ad21-4550c2f345d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:30.858353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 969/1934] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:49:30 2022] DELETE /identity/v3/projects/9e43b0850f04480ebc5c13622f97b1ea => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:49:30.862567 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def1ed2-7393-4dbf-92c9-8b1d3b9f7d9b service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:30.876724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def1ed2-7393-4dbf-92c9-8b1d3b9f7d9b service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:30.929201 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:30.955987 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f9b34-cc90-4d39-9881-163dafddd506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:30.957560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f9b34-cc90-4d39-9881-163dafddd506 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:30.958281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f9b34-cc90-4d39-9881-163dafddd506 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:30.958909 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f9b34-cc90-4d39-9881-163dafddd506 admin admin] PATH_INFO: `/v3/users/1f6d963ff4124a7ea8042b175143474b` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:30.960696 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f9b34-cc90-4d39-9881-163dafddd506 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f6d963ff4124a7ea8042b175143474b)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:30.964197 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f9b34-cc90-4d39-9881-163dafddd506 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:31.066327 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-407f9b34-cc90-4d39-9881-163dafddd506 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f6d963ff4124a7ea8042b175143474b'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:31.083484 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-407f9b34-cc90-4d39-9881-163dafddd506 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f6d963ff4124a7ea8042b175143474b'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:31.084262 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-407f9b34-cc90-4d39-9881-163dafddd506 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f6d963ff4124a7ea8042b175143474b'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:31.084885 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-407f9b34-cc90-4d39-9881-163dafddd506 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f6d963ff4124a7ea8042b175143474b'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:31.093169 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-407f9b34-cc90-4d39-9881-163dafddd506 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f6d963ff4124a7ea8042b175143474b'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:31.115704 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 966/1935] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:49:30 2022] DELETE /identity/v3/users/1f6d963ff4124a7ea8042b175143474b => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:49:31.448871 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cfdc39-8235-4944-ad21-4550c2f345d5 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:31.451566 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-40cfdc39-8235-4944-ad21-4550c2f345d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:31.475996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cfdc39-8235-4944-ad21-4550c2f345d5 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:31.545749 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a7df05-d110-48f9-8a3b-1f36a01c0f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:31.546683 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7df05-d110-48f9-8a3b-1f36a01c0f1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:31.546968 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7df05-d110-48f9-8a3b-1f36a01c0f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:31.547265 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7df05-d110-48f9-8a3b-1f36a01c0f1e admin admin] PATH_INFO: `/v3/projects/aadf14ba691d4ea483f095b19bbe734d` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:31.548537 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a7df05-d110-48f9-8a3b-1f36a01c0f1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=aadf14ba691d4ea483f095b19bbe734d)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:31.550506 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a7df05-d110-48f9-8a3b-1f36a01c0f1e admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:31.604747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-80a7df05-d110-48f9-8a3b-1f36a01c0f1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aadf14ba691d4ea483f095b19bbe734d', 'request_id': 'req-80a7df05-d110-48f9-8a3b-1f36a01c0f1e'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:31.610471 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-80a7df05-d110-48f9-8a3b-1f36a01c0f1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aadf14ba691d4ea483f095b19bbe734d', 'request_id': 'req-80a7df05-d110-48f9-8a3b-1f36a01c0f1e'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:31.619690 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-80a7df05-d110-48f9-8a3b-1f36a01c0f1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aadf14ba691d4ea483f095b19bbe734d', 'request_id': 'req-80a7df05-d110-48f9-8a3b-1f36a01c0f1e'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:31.632712 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-80a7df05-d110-48f9-8a3b-1f36a01c0f1e admin admin] The token cache is being invalidate because project aadf14ba691d4ea483f095b19bbe734d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:49:31.633012 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-80a7df05-d110-48f9-8a3b-1f36a01c0f1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:31.633798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-80a7df05-d110-48f9-8a3b-1f36a01c0f1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:31.635568 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 970/1936] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:49:31 2022] DELETE /identity/v3/projects/aadf14ba691d4ea483f095b19bbe734d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:49:31.640897 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856597b-0f75-4a66-b04f-41128dfcaeac None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:31.641302 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856597b-0f75-4a66-b04f-41128dfcaeac None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:31.641563 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856597b-0f75-4a66-b04f-41128dfcaeac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:31.654793 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a7df05-d110-48f9-8a3b-1f36a01c0f1e admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:31.657665 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-80a7df05-d110-48f9-8a3b-1f36a01c0f1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:31.662251 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-5856597b-0f75-4a66-b04f-41128dfcaeac None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:49:31.663690 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-5856597b-0f75-4a66-b04f-41128dfcaeac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:31.730928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a7df05-d110-48f9-8a3b-1f36a01c0f1e admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:31.736025 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bffeecc-1c77-4443-9302-d837097e299c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:31.736998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffeecc-1c77-4443-9302-d837097e299c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:31.737205 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffeecc-1c77-4443-9302-d837097e299c admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:31.737485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffeecc-1c77-4443-9302-d837097e299c admin admin] PATH_INFO: `/v3/users/6631ef3f6fd44e1083e54d76c0184301` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:31.738757 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bffeecc-1c77-4443-9302-d837097e299c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6631ef3f6fd44e1083e54d76c0184301)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:31.740752 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bffeecc-1c77-4443-9302-d837097e299c admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:31.748961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 967/1937] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:49:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:49:31.751945 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58d924-18ec-4667-8e57-516de391b811 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:31.752250 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58d924-18ec-4667-8e57-516de391b811 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:31.752537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58d924-18ec-4667-8e57-516de391b811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:31.773890 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-6c58d924-18ec-4667-8e57-516de391b811 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:49:31.775160 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-6c58d924-18ec-4667-8e57-516de391b811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:31.775670 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:31.785496 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 968/1938] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:49:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:49:31.790660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58d924-18ec-4667-8e57-516de391b811 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:31.802714 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58d924-18ec-4667-8e57-516de391b811 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:31.808254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c49166-a20d-40b7-b9fd-8ef00a6eae6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:31.809064 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c49166-a20d-40b7-b9fd-8ef00a6eae6e admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:31.809274 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c49166-a20d-40b7-b9fd-8ef00a6eae6e admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:31.809486 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c49166-a20d-40b7-b9fd-8ef00a6eae6e admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:31.810220 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c49166-a20d-40b7-b9fd-8ef00a6eae6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:49:31.810529 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c49166-a20d-40b7-b9fd-8ef00a6eae6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:31.812186 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c49166-a20d-40b7-b9fd-8ef00a6eae6e admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:31.814272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-6bffeecc-1c77-4443-9302-d837097e299c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6631ef3f6fd44e1083e54d76c0184301'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:31.835278 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 969/1939] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:49:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 14:49:31.837155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-6bffeecc-1c77-4443-9302-d837097e299c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6631ef3f6fd44e1083e54d76c0184301'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:31.837414 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c49166-a20d-40b7-b9fd-8ef00a6eae6e admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:31.837991 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-6bffeecc-1c77-4443-9302-d837097e299c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6631ef3f6fd44e1083e54d76c0184301'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:31.839494 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-6bffeecc-1c77-4443-9302-d837097e299c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6631ef3f6fd44e1083e54d76c0184301'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:31.848510 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c49166-a20d-40b7-b9fd-8ef00a6eae6e admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:31.857896 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-6bffeecc-1c77-4443-9302-d837097e299c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6631ef3f6fd44e1083e54d76c0184301'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:31.858801 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a736c-d57c-4e3c-a8ae-617e68158c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:31.860018 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a736c-d57c-4e3c-a8ae-617e68158c03 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:31.860502 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a736c-d57c-4e3c-a8ae-617e68158c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:31.861407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a736c-d57c-4e3c-a8ae-617e68158c03 admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:31.863249 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3a736c-d57c-4e3c-a8ae-617e68158c03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:31.866384 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3a736c-d57c-4e3c-a8ae-617e68158c03 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:31.875506 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 971/1940] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:49:31 2022] DELETE /identity/v3/users/6631ef3f6fd44e1083e54d76c0184301 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:49:31.879907 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b8580-e914-4530-b603-516b97295410 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:31.880265 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b8580-e914-4530-b603-516b97295410 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:31.880737 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b8580-e914-4530-b603-516b97295410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:31.901215 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 970/1941] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:49:31 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 14:49:31.905892 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a736c-d57c-4e3c-a8ae-617e68158c03 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:31.906514 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-1b7b8580-e914-4530-b603-516b97295410 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:49:31.907893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-1b7b8580-e914-4530-b603-516b97295410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:31.908372 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-2e3a736c-d57c-4e3c-a8ae-617e68158c03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:31.929816 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:31.990468 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a736c-d57c-4e3c-a8ae-617e68158c03 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:31.991759 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 972/1942] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:49:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:49:31.999636 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a9f4b1-dd84-4a70-ba67-d29fd60594fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:32.000312 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9f4b1-dd84-4a70-ba67-d29fd60594fe admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:32.000654 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9f4b1-dd84-4a70-ba67-d29fd60594fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:32.000933 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9f4b1-dd84-4a70-ba67-d29fd60594fe admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:32.001698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b8580-e914-4530-b603-516b97295410 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:32.001832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a9f4b1-dd84-4a70-ba67-d29fd60594fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:32.003272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-1b7b8580-e914-4530-b603-516b97295410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:32.003819 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a9f4b1-dd84-4a70-ba67-d29fd60594fe admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:32.025528 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 971/1943] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:49:31 2022] POST /identity/v3/users => generated 288 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 14:49:32.029923 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a9f4b1-dd84-4a70-ba67-d29fd60594fe admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:32.043407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a9f4b1-dd84-4a70-ba67-d29fd60594fe admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:32.049297 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86dd4a4-5166-45b4-ac3d-2f14dee7b92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:32.050245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86dd4a4-5166-45b4-ac3d-2f14dee7b92b admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:32.050694 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86dd4a4-5166-45b4-ac3d-2f14dee7b92b admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:32.051066 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86dd4a4-5166-45b4-ac3d-2f14dee7b92b admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:32.051844 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86dd4a4-5166-45b4-ac3d-2f14dee7b92b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:32.053519 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86dd4a4-5166-45b4-ac3d-2f14dee7b92b admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:32.068378 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 972/1944] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 14:49:32.069719 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86dd4a4-5166-45b4-ac3d-2f14dee7b92b admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:32.072714 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b8580-e914-4530-b603-516b97295410 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:32.076591 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db82b42-c725-4026-9ab2-90a224884463 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:32.079143 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db82b42-c725-4026-9ab2-90a224884463 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:32.079143 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db82b42-c725-4026-9ab2-90a224884463 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:32.079143 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db82b42-c725-4026-9ab2-90a224884463 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:32.085344 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86dd4a4-5166-45b4-ac3d-2f14dee7b92b admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:32.092371 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db82b42-c725-4026-9ab2-90a224884463 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:32.092568 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c0497-8a50-4058-b144-2f460f1e4827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:32.093185 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c0497-8a50-4058-b144-2f460f1e4827 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:32.093537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c0497-8a50-4058-b144-2f460f1e4827 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:32.093758 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c0497-8a50-4058-b144-2f460f1e4827 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:32.093991 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db82b42-c725-4026-9ab2-90a224884463 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:32.094536 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c0497-8a50-4058-b144-2f460f1e4827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:32.096347 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c0497-8a50-4058-b144-2f460f1e4827 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:32.109002 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 973/1945] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:49:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 14:49:32.111631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 973/1946] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:49:32.112347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db82b42-c725-4026-9ab2-90a224884463 service neutron] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:32.123895 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db82b42-c725-4026-9ab2-90a224884463 service neutron] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:32.131024 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba21d20-170a-4a38-b9b8-d1607c98ffce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:32.131926 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba21d20-170a-4a38-b9b8-d1607c98ffce admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:32.132214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba21d20-170a-4a38-b9b8-d1607c98ffce admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:32.132492 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba21d20-170a-4a38-b9b8-d1607c98ffce admin admin] PATH_INFO: `/v3/projects/a63e6efde0ec4d8fac265d68dd0f21a0/users/c010da5eaee94134b32ffe64c36861d5/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:32.150745 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba21d20-170a-4a38-b9b8-d1607c98ffce admin admin] RBAC: Authorizing `identity:create_grant(project_id=a63e6efde0ec4d8fac265d68dd0f21a0, user_id=c010da5eaee94134b32ffe64c36861d5, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:32.153454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba21d20-170a-4a38-b9b8-d1607c98ffce admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:32.173076 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 974/1947] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:49:32 2022] PUT /identity/v3/projects/a63e6efde0ec4d8fac265d68dd0f21a0/users/c010da5eaee94134b32ffe64c36861d5/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 14:49:32.183961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c818da-0d87-4701-9d3f-cc94e4b6155d None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:32.184256 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c818da-0d87-4701-9d3f-cc94e4b6155d None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:32.184522 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c818da-0d87-4701-9d3f-cc94e4b6155d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:32.216447 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-12c818da-0d87-4701-9d3f-cc94e4b6155d None None] MFA Rules not processed for user `c010da5eaee94134b32ffe64c36861d5`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:49:32.218410 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-12c818da-0d87-4701-9d3f-cc94e4b6155d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:32.301270 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 974/1948] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:49:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:49:32.306363 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba21d20-170a-4a38-b9b8-d1607c98ffce admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:32.328119 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:32.377288 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba21d20-170a-4a38-b9b8-d1607c98ffce admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:32.383293 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d45889-5bc8-4a4a-9c20-83459d40caff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:32.383293 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d45889-5bc8-4a4a-9c20-83459d40caff admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:32.383762 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d45889-5bc8-4a4a-9c20-83459d40caff admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:32.384061 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d45889-5bc8-4a4a-9c20-83459d40caff admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:32.384774 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d45889-5bc8-4a4a-9c20-83459d40caff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:32.387519 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d45889-5bc8-4a4a-9c20-83459d40caff admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:32.423568 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 975/1949] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:49:32 2022] POST /identity/v3/projects => generated 375 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 14:49:32.429263 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c818da-0d87-4701-9d3f-cc94e4b6155d None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:32.441863 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c818da-0d87-4701-9d3f-cc94e4b6155d None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:32.507680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba80f31-b2c5-4867-ae68-d190f3ad3b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:32.508473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba80f31-b2c5-4867-ae68-d190f3ad3b83 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:32.508826 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba80f31-b2c5-4867-ae68-d190f3ad3b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:32.509300 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba80f31-b2c5-4867-ae68-d190f3ad3b83 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:32.510174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba80f31-b2c5-4867-ae68-d190f3ad3b83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:32.512295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba80f31-b2c5-4867-ae68-d190f3ad3b83 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:32.538328 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 975/1950] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:49:32 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:49:32.539890 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d45889-5bc8-4a4a-9c20-83459d40caff admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:32.554116 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d45889-5bc8-4a4a-9c20-83459d40caff admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:32.568030 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de314b-7f25-4b12-ad68-89c4394c1e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:32.569130 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de314b-7f25-4b12-ad68-89c4394c1e5d admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:32.569693 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de314b-7f25-4b12-ad68-89c4394c1e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:32.570632 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de314b-7f25-4b12-ad68-89c4394c1e5d admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:32.571781 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de314b-7f25-4b12-ad68-89c4394c1e5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:32.574279 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de314b-7f25-4b12-ad68-89c4394c1e5d admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:32.597133 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 976/1951] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:49:32.597704 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba80f31-b2c5-4867-ae68-d190f3ad3b83 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:32.613490 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba80f31-b2c5-4867-ae68-d190f3ad3b83 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:32.614968 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:32.631967 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e0c59b-18d4-4b04-9480-130f8bf8d26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:32.633268 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0c59b-18d4-4b04-9480-130f8bf8d26f admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:32.633268 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0c59b-18d4-4b04-9480-130f8bf8d26f admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:32.633534 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0c59b-18d4-4b04-9480-130f8bf8d26f admin admin] PATH_INFO: `/v3/projects/39fe469725664b1ab6a4c6dd63d1a0c8/users/3479314733f44debbcc54f5c56811e60/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:32.666666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0c59b-18d4-4b04-9480-130f8bf8d26f admin admin] RBAC: Authorizing `identity:create_grant(project_id=39fe469725664b1ab6a4c6dd63d1a0c8, user_id=3479314733f44debbcc54f5c56811e60, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:32.672554 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0c59b-18d4-4b04-9480-130f8bf8d26f admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:32.698688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 976/1952] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:49:32 2022] PUT /identity/v3/projects/39fe469725664b1ab6a4c6dd63d1a0c8/users/3479314733f44debbcc54f5c56811e60/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:49:32.704032 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de314b-7f25-4b12-ad68-89c4394c1e5d admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:32.720691 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de314b-7f25-4b12-ad68-89c4394c1e5d admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:32.820966 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe274612-e211-4aa9-9c82-81c9dd0cdde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:32.822168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe274612-e211-4aa9-9c82-81c9dd0cdde4 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:32.822168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe274612-e211-4aa9-9c82-81c9dd0cdde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:32.822529 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe274612-e211-4aa9-9c82-81c9dd0cdde4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:32.822995 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe274612-e211-4aa9-9c82-81c9dd0cdde4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:32.824637 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe274612-e211-4aa9-9c82-81c9dd0cdde4 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:32.839085 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 977/1953] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 14:49:32.844028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e0c59b-18d4-4b04-9480-130f8bf8d26f admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:32.845290 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe274612-e211-4aa9-9c82-81c9dd0cdde4 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:32.847259 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-fe274612-e211-4aa9-9c82-81c9dd0cdde4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:32.858075 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e0c59b-18d4-4b04-9480-130f8bf8d26f admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:32.870255 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe274612-e211-4aa9-9c82-81c9dd0cdde4 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:32.882941 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6edee7-bbac-409c-a2c4-5e36bedf463a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:32.882941 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6edee7-bbac-409c-a2c4-5e36bedf463a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:32.882941 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6edee7-bbac-409c-a2c4-5e36bedf463a admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:32.884575 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6edee7-bbac-409c-a2c4-5e36bedf463a admin admin] PATH_INFO: `/v3/projects/29bdc06e08b34c7aaf8d8dffc296eb07` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:32.887205 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6edee7-bbac-409c-a2c4-5e36bedf463a admin admin] RBAC: Authorizing `identity:delete_project(project_id=29bdc06e08b34c7aaf8d8dffc296eb07)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:32.891648 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6edee7-bbac-409c-a2c4-5e36bedf463a admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:32.896462 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3c479a-2f6e-49ec-9e33-fc72d8181287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:32.897393 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c479a-2f6e-49ec-9e33-fc72d8181287 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:32.897695 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c479a-2f6e-49ec-9e33-fc72d8181287 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:32.898105 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c479a-2f6e-49ec-9e33-fc72d8181287 admin admin] PATH_INFO: `/v3/projects/39fe469725664b1ab6a4c6dd63d1a0c8/users/3479314733f44debbcc54f5c56811e60/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:32.900942 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3c479a-2f6e-49ec-9e33-fc72d8181287 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39fe469725664b1ab6a4c6dd63d1a0c8, user_id=3479314733f44debbcc54f5c56811e60, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:32.909658 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3c479a-2f6e-49ec-9e33-fc72d8181287 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:32.942130 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 977/1954] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:49:32 2022] PUT /identity/v3/projects/39fe469725664b1ab6a4c6dd63d1a0c8/users/3479314733f44debbcc54f5c56811e60/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:49:32.953666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73992673-6cbc-4671-8b00-334c5c82c26d None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:32.954096 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73992673-6cbc-4671-8b00-334c5c82c26d None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:32.954289 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73992673-6cbc-4671-8b00-334c5c82c26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:32.984124 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fe6edee7-bbac-409c-a2c4-5e36bedf463a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29bdc06e08b34c7aaf8d8dffc296eb07', 'request_id': 'req-fe6edee7-bbac-409c-a2c4-5e36bedf463a'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:32.991202 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fe6edee7-bbac-409c-a2c4-5e36bedf463a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29bdc06e08b34c7aaf8d8dffc296eb07', 'request_id': 'req-fe6edee7-bbac-409c-a2c4-5e36bedf463a'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:33.003423 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-73992673-6cbc-4671-8b00-334c5c82c26d None None] MFA Rules not processed for user `3479314733f44debbcc54f5c56811e60`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:49:33.004906 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-73992673-6cbc-4671-8b00-334c5c82c26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:33.011554 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fe6edee7-bbac-409c-a2c4-5e36bedf463a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29bdc06e08b34c7aaf8d8dffc296eb07', 'request_id': 'req-fe6edee7-bbac-409c-a2c4-5e36bedf463a'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:33.029484 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fe6edee7-bbac-409c-a2c4-5e36bedf463a admin admin] The token cache is being invalidate because project 29bdc06e08b34c7aaf8d8dffc296eb07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:49:33.029822 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fe6edee7-bbac-409c-a2c4-5e36bedf463a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:33.030837 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fe6edee7-bbac-409c-a2c4-5e36bedf463a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:33.033639 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 978/1955] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:49:32 2022] DELETE /identity/v3/projects/29bdc06e08b34c7aaf8d8dffc296eb07 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:49:33.039808 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a238d80-82ed-4f55-9e14-3622ade65de3 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:33.040173 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a238d80-82ed-4f55-9e14-3622ade65de3 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:33.040522 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a238d80-82ed-4f55-9e14-3622ade65de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:33.068274 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-0a238d80-82ed-4f55-9e14-3622ade65de3 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:49:33.070339 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-0a238d80-82ed-4f55-9e14-3622ade65de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:33.083713 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:33.134569 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 978/1956] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:49:32 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:49:33.152137 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73992673-6cbc-4671-8b00-334c5c82c26d None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:33.154369 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-73992673-6cbc-4671-8b00-334c5c82c26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:33.187229 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 979/1957] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:49:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:49:33.193212 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61597736-87ed-4ab9-9d6b-4d90c513266b None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:33.193565 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61597736-87ed-4ab9-9d6b-4d90c513266b None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:33.193897 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61597736-87ed-4ab9-9d6b-4d90c513266b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:33.224597 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-61597736-87ed-4ab9-9d6b-4d90c513266b None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:49:33.225825 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-61597736-87ed-4ab9-9d6b-4d90c513266b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:33.230393 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73992673-6cbc-4671-8b00-334c5c82c26d None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:33.235042 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ab2d8-350a-4134-85b1-ddc0447a9534 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:33.235839 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ab2d8-350a-4134-85b1-ddc0447a9534 service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:33.236248 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ab2d8-350a-4134-85b1-ddc0447a9534 service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:33.236632 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ab2d8-350a-4134-85b1-ddc0447a9534 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:33.238082 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 980/1958] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:49:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:49:33.238858 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-742ab2d8-350a-4134-85b1-ddc0447a9534 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:33.239183 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61597736-87ed-4ab9-9d6b-4d90c513266b None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:33.248963 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61597736-87ed-4ab9-9d6b-4d90c513266b None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:33.262623 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04cf73c-19d9-448c-8e60-f3c26aa2703d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:33.263867 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cf73c-19d9-448c-8e60-f3c26aa2703d admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:33.263867 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cf73c-19d9-448c-8e60-f3c26aa2703d admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:33.264071 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cf73c-19d9-448c-8e60-f3c26aa2703d admin admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:33.265252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04cf73c-19d9-448c-8e60-f3c26aa2703d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:49:33.265581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04cf73c-19d9-448c-8e60-f3c26aa2703d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:33.267589 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04cf73c-19d9-448c-8e60-f3c26aa2703d admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:33.286473 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 981/1959] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:49:33.287963 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04cf73c-19d9-448c-8e60-f3c26aa2703d admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:33.299676 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04cf73c-19d9-448c-8e60-f3c26aa2703d admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:33.307077 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca8c9de-8574-40db-85b2-be35747b9f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:33.307923 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8c9de-8574-40db-85b2-be35747b9f38 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:33.308610 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8c9de-8574-40db-85b2-be35747b9f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:33.308938 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8c9de-8574-40db-85b2-be35747b9f38 admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:33.309812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca8c9de-8574-40db-85b2-be35747b9f38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:33.312884 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca8c9de-8574-40db-85b2-be35747b9f38 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:33.328098 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742ab2d8-350a-4134-85b1-ddc0447a9534 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:33.329633 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742ab2d8-350a-4134-85b1-ddc0447a9534 service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:33.342698 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 979/1960] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:49:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 14:49:33.346016 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 982/1961] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:49:33 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:49:33.350450 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ab2d8-350a-4134-85b1-ddc0447a9534 service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:33.364688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ab2d8-350a-4134-85b1-ddc0447a9534 service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:33.369188 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:33.444387 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919b233-1c00-4069-906b-1a8f19817a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:33.445152 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919b233-1c00-4069-906b-1a8f19817a2f admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:33.445507 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919b233-1c00-4069-906b-1a8f19817a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:33.445885 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919b233-1c00-4069-906b-1a8f19817a2f admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:33.446685 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2919b233-1c00-4069-906b-1a8f19817a2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:33.449053 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2919b233-1c00-4069-906b-1a8f19817a2f admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:33.491160 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 980/1962] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:49:33 2022] POST /identity/v3/users => generated 292 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 14:49:33.493402 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca8c9de-8574-40db-85b2-be35747b9f38 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:33.505128 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca8c9de-8574-40db-85b2-be35747b9f38 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:33.511258 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745bcf3-fb18-4f9b-975c-53893e092139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:33.512269 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745bcf3-fb18-4f9b-975c-53893e092139 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:33.513652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745bcf3-fb18-4f9b-975c-53893e092139 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:33.513929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745bcf3-fb18-4f9b-975c-53893e092139 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:33.514836 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4745bcf3-fb18-4f9b-975c-53893e092139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:33.516873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4745bcf3-fb18-4f9b-975c-53893e092139 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:33.534796 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 983/1963] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:49:33.535341 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919b233-1c00-4069-906b-1a8f19817a2f admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:33.546419 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919b233-1c00-4069-906b-1a8f19817a2f admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:33.552291 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff179f7-f931-4c99-bb3b-e2fa232bdb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:33.554241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff179f7-f931-4c99-bb3b-e2fa232bdb66 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:33.554686 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff179f7-f931-4c99-bb3b-e2fa232bdb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:33.555167 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff179f7-f931-4c99-bb3b-e2fa232bdb66 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:33.556615 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff179f7-f931-4c99-bb3b-e2fa232bdb66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:33.559437 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff179f7-f931-4c99-bb3b-e2fa232bdb66 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:33.575580 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 981/1964] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 14:49:33.577098 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745bcf3-fb18-4f9b-975c-53893e092139 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:33.591678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745bcf3-fb18-4f9b-975c-53893e092139 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:33.597402 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eebd7bc-3c14-4995-8fa9-de868dfeb092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:33.598700 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebd7bc-3c14-4995-8fa9-de868dfeb092 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:33.598928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebd7bc-3c14-4995-8fa9-de868dfeb092 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:33.599268 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebd7bc-3c14-4995-8fa9-de868dfeb092 admin admin] PATH_INFO: `/v3/projects/a4f0bbbe0b1245f0a0006d26a0e21ff8/users/8b03e9f615214e5384303399a13489ee/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:33.618075 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eebd7bc-3c14-4995-8fa9-de868dfeb092 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4f0bbbe0b1245f0a0006d26a0e21ff8, user_id=8b03e9f615214e5384303399a13489ee, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:33.622145 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eebd7bc-3c14-4995-8fa9-de868dfeb092 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:33.642463 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 984/1965] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:49:33 2022] PUT /identity/v3/projects/a4f0bbbe0b1245f0a0006d26a0e21ff8/users/8b03e9f615214e5384303399a13489ee/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:49:33.653503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e4868-7d62-404f-b323-07a269cce12d None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:33.653841 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e4868-7d62-404f-b323-07a269cce12d None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:33.654127 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e4868-7d62-404f-b323-07a269cce12d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:33.692372 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-206e4868-7d62-404f-b323-07a269cce12d None None] MFA Rules not processed for user `8b03e9f615214e5384303399a13489ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:49:33.694158 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-206e4868-7d62-404f-b323-07a269cce12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:33.781138 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 982/1966] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:49:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:49:33.785898 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eebd7bc-3c14-4995-8fa9-de868dfeb092 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:33.863407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eebd7bc-3c14-4995-8fa9-de868dfeb092 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:33.871138 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83be8e7-165f-4fc9-b8ff-53e02c59d176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:33.871832 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83be8e7-165f-4fc9-b8ff-53e02c59d176 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:33.872143 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83be8e7-165f-4fc9-b8ff-53e02c59d176 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:33.872421 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83be8e7-165f-4fc9-b8ff-53e02c59d176 admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:33.873142 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83be8e7-165f-4fc9-b8ff-53e02c59d176 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:33.877167 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83be8e7-165f-4fc9-b8ff-53e02c59d176 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:33.926205 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 985/1967] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:49:33 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 14:49:33.927236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e4868-7d62-404f-b323-07a269cce12d None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:33.940506 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e4868-7d62-404f-b323-07a269cce12d None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:34.022864 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627c2d5-0dce-48c8-a35e-a5a96bd84ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:34.024664 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627c2d5-0dce-48c8-a35e-a5a96bd84ec5 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:34.024985 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627c2d5-0dce-48c8-a35e-a5a96bd84ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:34.025278 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627c2d5-0dce-48c8-a35e-a5a96bd84ec5 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:34.026187 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b627c2d5-0dce-48c8-a35e-a5a96bd84ec5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:34.028593 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b627c2d5-0dce-48c8-a35e-a5a96bd84ec5 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:34.060426 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 983/1968] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:49:33 2022] POST /identity/v3/users => generated 293 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:49:34.063389 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83be8e7-165f-4fc9-b8ff-53e02c59d176 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:34.077104 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83be8e7-165f-4fc9-b8ff-53e02c59d176 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:34.084222 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdaa825-057d-4604-aff8-1c69e49078f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:34.084222 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdaa825-057d-4604-aff8-1c69e49078f2 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:34.084222 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdaa825-057d-4604-aff8-1c69e49078f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:34.084222 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdaa825-057d-4604-aff8-1c69e49078f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:34.084873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdaa825-057d-4604-aff8-1c69e49078f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:34.086656 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdaa825-057d-4604-aff8-1c69e49078f2 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:34.102736 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 986/1969] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 14:49:34.104725 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627c2d5-0dce-48c8-a35e-a5a96bd84ec5 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:34.111950 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:34.117377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627c2d5-0dce-48c8-a35e-a5a96bd84ec5 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:34.124605 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf4d3b-554d-42a1-a603-77b4a4a93ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:34.126528 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf4d3b-554d-42a1-a603-77b4a4a93ec1 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:34.127246 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf4d3b-554d-42a1-a603-77b4a4a93ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:34.127872 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf4d3b-554d-42a1-a603-77b4a4a93ec1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:34.134374 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf4d3b-554d-42a1-a603-77b4a4a93ec1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:34.136182 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf4d3b-554d-42a1-a603-77b4a4a93ec1 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:34.154028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 984/1970] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:49:34.157794 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdaa825-057d-4604-aff8-1c69e49078f2 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:34.184293 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdaa825-057d-4604-aff8-1c69e49078f2 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:34.200051 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e2834-1e55-4ab3-8a29-a8af556bfc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:34.201346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e2834-1e55-4ab3-8a29-a8af556bfc82 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:34.201637 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e2834-1e55-4ab3-8a29-a8af556bfc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:34.201972 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e2834-1e55-4ab3-8a29-a8af556bfc82 admin admin] PATH_INFO: `/v3/projects/0fed1489e9784c6fb516d31822afc99d/users/0249ff3b0ed445b2821f3cff6c6702ad/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:34.214852 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:34.232329 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e2834-1e55-4ab3-8a29-a8af556bfc82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fed1489e9784c6fb516d31822afc99d, user_id=0249ff3b0ed445b2821f3cff6c6702ad, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:34.235515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e2834-1e55-4ab3-8a29-a8af556bfc82 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:34.273740 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 987/1971] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:49:34 2022] PUT /identity/v3/projects/0fed1489e9784c6fb516d31822afc99d/users/0249ff3b0ed445b2821f3cff6c6702ad/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:49:34.278119 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b96f7-846e-4f91-baf3-fbc3f16378ae None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:34.278333 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b96f7-846e-4f91-baf3-fbc3f16378ae None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:34.278536 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b96f7-846e-4f91-baf3-fbc3f16378ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:34.315970 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-f57b96f7-846e-4f91-baf3-fbc3f16378ae None None] MFA Rules not processed for user `0249ff3b0ed445b2821f3cff6c6702ad`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:49:34.326166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-f57b96f7-846e-4f91-baf3-fbc3f16378ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:34.436987 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 985/1972] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:49:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:49:34.443713 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e2834-1e55-4ab3-8a29-a8af556bfc82 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:34.567647 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e2834-1e55-4ab3-8a29-a8af556bfc82 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:34.579377 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78496346-9ddc-4aef-b626-6ce98eec4865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:34.580785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78496346-9ddc-4aef-b626-6ce98eec4865 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:34.580785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78496346-9ddc-4aef-b626-6ce98eec4865 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:34.581152 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78496346-9ddc-4aef-b626-6ce98eec4865 admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:34.582226 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78496346-9ddc-4aef-b626-6ce98eec4865 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:34.586456 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78496346-9ddc-4aef-b626-6ce98eec4865 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:34.657875 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b96f7-846e-4f91-baf3-fbc3f16378ae None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:34.658875 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 988/1973] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:49:34 2022] POST /identity/v3/projects => generated 383 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 14:49:34.672353 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b96f7-846e-4f91-baf3-fbc3f16378ae None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:34.748678 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34566d8f-45a8-4975-a2a7-343695fb9f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:34.750465 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34566d8f-45a8-4975-a2a7-343695fb9f52 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:34.750465 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34566d8f-45a8-4975-a2a7-343695fb9f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:34.750465 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34566d8f-45a8-4975-a2a7-343695fb9f52 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:34.752178 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34566d8f-45a8-4975-a2a7-343695fb9f52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:34.754526 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34566d8f-45a8-4975-a2a7-343695fb9f52 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:34.786356 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 986/1974] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:49:34 2022] POST /identity/v3/users => generated 298 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:49:34.791312 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78496346-9ddc-4aef-b626-6ce98eec4865 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:34.803716 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78496346-9ddc-4aef-b626-6ce98eec4865 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:34.808962 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbeb4c4-543d-469e-a5dc-6fd09bd6ea71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:34.809923 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbeb4c4-543d-469e-a5dc-6fd09bd6ea71 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:34.810301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbeb4c4-543d-469e-a5dc-6fd09bd6ea71 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:34.810649 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbeb4c4-543d-469e-a5dc-6fd09bd6ea71 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:34.811644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbeb4c4-543d-469e-a5dc-6fd09bd6ea71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:34.813698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbeb4c4-543d-469e-a5dc-6fd09bd6ea71 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:34.826536 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 989/1975] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 14:49:34.831533 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34566d8f-45a8-4975-a2a7-343695fb9f52 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:34.845344 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34566d8f-45a8-4975-a2a7-343695fb9f52 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:34.851514 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d6cb8-ee84-4798-a553-1edfba9a3309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:34.853386 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d6cb8-ee84-4798-a553-1edfba9a3309 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:34.853760 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d6cb8-ee84-4798-a553-1edfba9a3309 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:34.854110 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d6cb8-ee84-4798-a553-1edfba9a3309 admin admin] PATH_INFO: `/v3/projects/1f53086ba24643d5b406692708ff8c70/users/8ebc84cae6104781a42a3756942abcd5/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:34.878666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d6cb8-ee84-4798-a553-1edfba9a3309 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f53086ba24643d5b406692708ff8c70, user_id=8ebc84cae6104781a42a3756942abcd5, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:34.883843 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d6cb8-ee84-4798-a553-1edfba9a3309 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:34.914781 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbeb4c4-543d-469e-a5dc-6fd09bd6ea71 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:34.915556 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 987/1976] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:49:34 2022] PUT /identity/v3/projects/1f53086ba24643d5b406692708ff8c70/users/8ebc84cae6104781a42a3756942abcd5/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:49:34.927409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbeb4c4-543d-469e-a5dc-6fd09bd6ea71 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:34.999746 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3fc7ff-be93-461a-bf08-59bb5209b846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:35.001307 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3fc7ff-be93-461a-bf08-59bb5209b846 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:35.001307 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3fc7ff-be93-461a-bf08-59bb5209b846 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:35.001307 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3fc7ff-be93-461a-bf08-59bb5209b846 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:35.002191 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3fc7ff-be93-461a-bf08-59bb5209b846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:35.003970 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3fc7ff-be93-461a-bf08-59bb5209b846 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:35.025265 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d6cb8-ee84-4798-a553-1edfba9a3309 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:35.026989 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 990/1977] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:49:35.039188 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d6cb8-ee84-4798-a553-1edfba9a3309 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:35.057844 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cde551-7cb4-4bc7-9736-cd61f7fbee77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:35.058788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cde551-7cb4-4bc7-9736-cd61f7fbee77 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:35.059112 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cde551-7cb4-4bc7-9736-cd61f7fbee77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:35.059405 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cde551-7cb4-4bc7-9736-cd61f7fbee77 admin admin] PATH_INFO: `/v3/projects/1f53086ba24643d5b406692708ff8c70/users/8ebc84cae6104781a42a3756942abcd5/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:35.068105 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cde551-7cb4-4bc7-9736-cd61f7fbee77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f53086ba24643d5b406692708ff8c70, user_id=8ebc84cae6104781a42a3756942abcd5, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:35.082362 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cde551-7cb4-4bc7-9736-cd61f7fbee77 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:35.111994 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 988/1978] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:49:35 2022] PUT /identity/v3/projects/1f53086ba24643d5b406692708ff8c70/users/8ebc84cae6104781a42a3756942abcd5/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:49:35.124287 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeddc3c-542e-4e87-b5aa-ad4d15cef8c2 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:35.124570 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeddc3c-542e-4e87-b5aa-ad4d15cef8c2 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:35.124883 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeddc3c-542e-4e87-b5aa-ad4d15cef8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:35.165915 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-feeddc3c-542e-4e87-b5aa-ad4d15cef8c2 None None] MFA Rules not processed for user `8ebc84cae6104781a42a3756942abcd5`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:49:35.167239 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-feeddc3c-542e-4e87-b5aa-ad4d15cef8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:35.225426 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:35.263097 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 991/1979] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:49:35 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:49:35.276977 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cde551-7cb4-4bc7-9736-cd61f7fbee77 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:35.348856 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:35.420886 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cde551-7cb4-4bc7-9736-cd61f7fbee77 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:35.441299 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3343496a-014b-4a51-8ac8-112a3dc7c0df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:35.444323 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343496a-014b-4a51-8ac8-112a3dc7c0df service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:35.444691 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343496a-014b-4a51-8ac8-112a3dc7c0df service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:35.445164 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343496a-014b-4a51-8ac8-112a3dc7c0df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:35.588757 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3343496a-014b-4a51-8ac8-112a3dc7c0df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:35.589902 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3343496a-014b-4a51-8ac8-112a3dc7c0df service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:35.604972 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 989/1980] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:49:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:49:40.312044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeddc3c-542e-4e87-b5aa-ad4d15cef8c2 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:40.322474 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeddc3c-542e-4e87-b5aa-ad4d15cef8c2 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:40.326737 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96faff74-8db3-43e9-8493-41a05ab5e0d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:40.327448 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96faff74-8db3-43e9-8493-41a05ab5e0d9 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:40.327733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96faff74-8db3-43e9-8493-41a05ab5e0d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:40.328007 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96faff74-8db3-43e9-8493-41a05ab5e0d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:40.398852 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96faff74-8db3-43e9-8493-41a05ab5e0d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:40.400258 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96faff74-8db3-43e9-8493-41a05ab5e0d9 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:40.412378 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 992/1981] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:49:40 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:49:41.527393 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3343496a-014b-4a51-8ac8-112a3dc7c0df service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:41.528889 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-3343496a-014b-4a51-8ac8-112a3dc7c0df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:41.540768 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3343496a-014b-4a51-8ac8-112a3dc7c0df service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:41.545723 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83272fe2-429c-4ebd-9f47-8e8d9919e06b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:41.546523 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83272fe2-429c-4ebd-9f47-8e8d9919e06b service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:41.546824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83272fe2-429c-4ebd-9f47-8e8d9919e06b service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:41.547101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83272fe2-429c-4ebd-9f47-8e8d9919e06b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:41.609611 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83272fe2-429c-4ebd-9f47-8e8d9919e06b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:41.611040 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83272fe2-429c-4ebd-9f47-8e8d9919e06b service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:41.620546 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 990/1982] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:49:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:49:44.589904 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96faff74-8db3-43e9-8493-41a05ab5e0d9 service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:44.598945 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96faff74-8db3-43e9-8493-41a05ab5e0d9 service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:44.603084 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322a8f3-7cb7-40f8-b538-27bc9221777f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:44.603848 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322a8f3-7cb7-40f8-b538-27bc9221777f service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:44.604204 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322a8f3-7cb7-40f8-b538-27bc9221777f service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:44.604546 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322a8f3-7cb7-40f8-b538-27bc9221777f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:44.614710 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8322a8f3-7cb7-40f8-b538-27bc9221777f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:44.616496 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8322a8f3-7cb7-40f8-b538-27bc9221777f service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:44.626877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 993/1983] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:49:44 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:49:51.616269 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=60 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:51.990867 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83272fe2-429c-4ebd-9f47-8e8d9919e06b service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:52.057229 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83272fe2-429c-4ebd-9f47-8e8d9919e06b service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:52.061948 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee33fc9-7028-444f-800f-ed61926d5d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:52.062869 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee33fc9-7028-444f-800f-ed61926d5d29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:52.063178 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee33fc9-7028-444f-800f-ed61926d5d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:52.063456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee33fc9-7028-444f-800f-ed61926d5d29 admin admin] PATH_INFO: `/v3/users/8b03e9f615214e5384303399a13489ee` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:52.064668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee33fc9-7028-444f-800f-ed61926d5d29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b03e9f615214e5384303399a13489ee)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:52.066655 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee33fc9-7028-444f-800f-ed61926d5d29 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:52.117020 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-0ee33fc9-7028-444f-800f-ed61926d5d29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b03e9f615214e5384303399a13489ee'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:52.129212 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-0ee33fc9-7028-444f-800f-ed61926d5d29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b03e9f615214e5384303399a13489ee'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:52.129964 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-0ee33fc9-7028-444f-800f-ed61926d5d29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b03e9f615214e5384303399a13489ee'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:52.130590 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-0ee33fc9-7028-444f-800f-ed61926d5d29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b03e9f615214e5384303399a13489ee'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:52.137485 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-0ee33fc9-7028-444f-800f-ed61926d5d29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b03e9f615214e5384303399a13489ee'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:52.156644 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 991/1984] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:49:51 2022] DELETE /identity/v3/users/8b03e9f615214e5384303399a13489ee => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:49:52.168986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd9fba-fb4c-4d01-b8fd-f262c8bc0ecf None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:52.169458 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd9fba-fb4c-4d01-b8fd-f262c8bc0ecf None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:52.169557 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd9fba-fb4c-4d01-b8fd-f262c8bc0ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:52.192598 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-63dd9fba-fb4c-4d01-b8fd-f262c8bc0ecf None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:49:52.193928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-63dd9fba-fb4c-4d01-b8fd-f262c8bc0ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:52.267840 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 994/1985] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:49:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:49:52.278940 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee33fc9-7028-444f-800f-ed61926d5d29 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:52.281122 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-0ee33fc9-7028-444f-800f-ed61926d5d29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:52.360132 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee33fc9-7028-444f-800f-ed61926d5d29 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:52.365607 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e2e95-2e5d-46cc-b1a6-046bf67c109d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:52.366759 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e2e95-2e5d-46cc-b1a6-046bf67c109d service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:52.367067 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e2e95-2e5d-46cc-b1a6-046bf67c109d service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:52.367362 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e2e95-2e5d-46cc-b1a6-046bf67c109d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:52.378534 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e2e95-2e5d-46cc-b1a6-046bf67c109d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:52.380034 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e2e95-2e5d-46cc-b1a6-046bf67c109d service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:52.391283 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 992/1986] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:49:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:49:52.712264 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dd9fba-fb4c-4d01-b8fd-f262c8bc0ecf None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:52.713845 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-63dd9fba-fb4c-4d01-b8fd-f262c8bc0ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:52.728654 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dd9fba-fb4c-4d01-b8fd-f262c8bc0ecf None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:52.737948 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8f6cd-4dfd-45ad-a4bf-55214b72cae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:52.739030 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8f6cd-4dfd-45ad-a4bf-55214b72cae7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:52.739607 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8f6cd-4dfd-45ad-a4bf-55214b72cae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:52.740219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8f6cd-4dfd-45ad-a4bf-55214b72cae7 admin admin] PATH_INFO: `/v3/projects/a4f0bbbe0b1245f0a0006d26a0e21ff8` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:52.741473 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8f6cd-4dfd-45ad-a4bf-55214b72cae7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4f0bbbe0b1245f0a0006d26a0e21ff8)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:52.743484 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8f6cd-4dfd-45ad-a4bf-55214b72cae7 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:52.797579 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-e3b8f6cd-4dfd-45ad-a4bf-55214b72cae7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4f0bbbe0b1245f0a0006d26a0e21ff8', 'request_id': 'req-e3b8f6cd-4dfd-45ad-a4bf-55214b72cae7'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:52.803957 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-e3b8f6cd-4dfd-45ad-a4bf-55214b72cae7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4f0bbbe0b1245f0a0006d26a0e21ff8', 'request_id': 'req-e3b8f6cd-4dfd-45ad-a4bf-55214b72cae7'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:52.805620 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:52.812803 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-e3b8f6cd-4dfd-45ad-a4bf-55214b72cae7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4f0bbbe0b1245f0a0006d26a0e21ff8', 'request_id': 'req-e3b8f6cd-4dfd-45ad-a4bf-55214b72cae7'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:52.824927 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-e3b8f6cd-4dfd-45ad-a4bf-55214b72cae7 admin admin] The token cache is being invalidate because project a4f0bbbe0b1245f0a0006d26a0e21ff8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:49:52.825243 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-e3b8f6cd-4dfd-45ad-a4bf-55214b72cae7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:52.825911 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-e3b8f6cd-4dfd-45ad-a4bf-55214b72cae7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:52.827597 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 995/1987] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:49:52 2022] DELETE /identity/v3/projects/a4f0bbbe0b1245f0a0006d26a0e21ff8 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 14:49:52.831810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e2e95-2e5d-46cc-b1a6-046bf67c109d service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:52.842370 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e2e95-2e5d-46cc-b1a6-046bf67c109d service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:52.862035 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:52.909230 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b9b072-63cd-460c-87fc-cc1d17431ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:52.910085 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9b072-63cd-460c-87fc-cc1d17431ad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:52.910399 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9b072-63cd-460c-87fc-cc1d17431ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:52.910678 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9b072-63cd-460c-87fc-cc1d17431ad2 admin admin] PATH_INFO: `/v3/users/0249ff3b0ed445b2821f3cff6c6702ad` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:52.912045 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b9b072-63cd-460c-87fc-cc1d17431ad2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0249ff3b0ed445b2821f3cff6c6702ad)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:52.914034 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b9b072-63cd-460c-87fc-cc1d17431ad2 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:52.967735 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-61b9b072-63cd-460c-87fc-cc1d17431ad2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0249ff3b0ed445b2821f3cff6c6702ad'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:52.981463 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-61b9b072-63cd-460c-87fc-cc1d17431ad2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0249ff3b0ed445b2821f3cff6c6702ad'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:52.982229 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-61b9b072-63cd-460c-87fc-cc1d17431ad2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0249ff3b0ed445b2821f3cff6c6702ad'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:52.982871 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-61b9b072-63cd-460c-87fc-cc1d17431ad2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0249ff3b0ed445b2821f3cff6c6702ad'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:52.990042 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-61b9b072-63cd-460c-87fc-cc1d17431ad2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0249ff3b0ed445b2821f3cff6c6702ad'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:53.003665 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 993/1988] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:49:52 2022] DELETE /identity/v3/users/0249ff3b0ed445b2821f3cff6c6702ad => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:49:53.377475 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8f6cd-4dfd-45ad-a4bf-55214b72cae7 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:53.379221 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-e3b8f6cd-4dfd-45ad-a4bf-55214b72cae7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:53.393538 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8f6cd-4dfd-45ad-a4bf-55214b72cae7 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:53.462745 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b3e82-bf6f-41ce-9a6a-63b9825fb8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:53.463671 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b3e82-bf6f-41ce-9a6a-63b9825fb8c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:53.463954 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b3e82-bf6f-41ce-9a6a-63b9825fb8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:53.464231 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b3e82-bf6f-41ce-9a6a-63b9825fb8c4 admin admin] PATH_INFO: `/v3/projects/0fed1489e9784c6fb516d31822afc99d` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:53.465487 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375b3e82-bf6f-41ce-9a6a-63b9825fb8c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fed1489e9784c6fb516d31822afc99d)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:53.467475 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375b3e82-bf6f-41ce-9a6a-63b9825fb8c4 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:53.516791 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-375b3e82-bf6f-41ce-9a6a-63b9825fb8c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fed1489e9784c6fb516d31822afc99d', 'request_id': 'req-375b3e82-bf6f-41ce-9a6a-63b9825fb8c4'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:53.522632 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-375b3e82-bf6f-41ce-9a6a-63b9825fb8c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fed1489e9784c6fb516d31822afc99d', 'request_id': 'req-375b3e82-bf6f-41ce-9a6a-63b9825fb8c4'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:53.531950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-375b3e82-bf6f-41ce-9a6a-63b9825fb8c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fed1489e9784c6fb516d31822afc99d', 'request_id': 'req-375b3e82-bf6f-41ce-9a6a-63b9825fb8c4'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:53.544774 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-375b3e82-bf6f-41ce-9a6a-63b9825fb8c4 admin admin] The token cache is being invalidate because project 0fed1489e9784c6fb516d31822afc99d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:49:53.545071 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-375b3e82-bf6f-41ce-9a6a-63b9825fb8c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:53.545844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-375b3e82-bf6f-41ce-9a6a-63b9825fb8c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:53.547703 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 996/1989] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:49:53 2022] DELETE /identity/v3/projects/0fed1489e9784c6fb516d31822afc99d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:49:53.552057 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b9b072-63cd-460c-87fc-cc1d17431ad2 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:53.553902 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-61b9b072-63cd-460c-87fc-cc1d17431ad2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:53.568318 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b9b072-63cd-460c-87fc-cc1d17431ad2 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:53.631715 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f6696-4dea-426f-94c7-30e59e2bbcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:53.633798 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f6696-4dea-426f-94c7-30e59e2bbcdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:53.634087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f6696-4dea-426f-94c7-30e59e2bbcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:53.634361 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f6696-4dea-426f-94c7-30e59e2bbcdd admin admin] PATH_INFO: `/v3/users/8ebc84cae6104781a42a3756942abcd5` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:53.635708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828f6696-4dea-426f-94c7-30e59e2bbcdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ebc84cae6104781a42a3756942abcd5)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:53.637689 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828f6696-4dea-426f-94c7-30e59e2bbcdd admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:53.705608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:53.709248 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-828f6696-4dea-426f-94c7-30e59e2bbcdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ebc84cae6104781a42a3756942abcd5'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:53.727946 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-828f6696-4dea-426f-94c7-30e59e2bbcdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ebc84cae6104781a42a3756942abcd5'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:53.728694 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-828f6696-4dea-426f-94c7-30e59e2bbcdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ebc84cae6104781a42a3756942abcd5'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:53.729305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-828f6696-4dea-426f-94c7-30e59e2bbcdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ebc84cae6104781a42a3756942abcd5'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:53.736789 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-828f6696-4dea-426f-94c7-30e59e2bbcdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ebc84cae6104781a42a3756942abcd5'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:53.754265 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 994/1990] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:49:53 2022] DELETE /identity/v3/users/8ebc84cae6104781a42a3756942abcd5 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:49:54.051353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b3e82-bf6f-41ce-9a6a-63b9825fb8c4 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:54.052913 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-375b3e82-bf6f-41ce-9a6a-63b9825fb8c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:54.068017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b3e82-bf6f-41ce-9a6a-63b9825fb8c4 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:54.136878 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3f1d2-b6eb-4f07-a632-7df878f9766a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:54.137784 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3f1d2-b6eb-4f07-a632-7df878f9766a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:54.138085 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3f1d2-b6eb-4f07-a632-7df878f9766a admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:54.138347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3f1d2-b6eb-4f07-a632-7df878f9766a admin admin] PATH_INFO: `/v3/projects/1f53086ba24643d5b406692708ff8c70` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:54.139629 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f3f1d2-b6eb-4f07-a632-7df878f9766a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f53086ba24643d5b406692708ff8c70)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:54.141549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f3f1d2-b6eb-4f07-a632-7df878f9766a admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:54.164720 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:54.193062 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-b4f3f1d2-b6eb-4f07-a632-7df878f9766a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f53086ba24643d5b406692708ff8c70', 'request_id': 'req-b4f3f1d2-b6eb-4f07-a632-7df878f9766a'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:54.198770 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-b4f3f1d2-b6eb-4f07-a632-7df878f9766a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f53086ba24643d5b406692708ff8c70', 'request_id': 'req-b4f3f1d2-b6eb-4f07-a632-7df878f9766a'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:54.208244 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-b4f3f1d2-b6eb-4f07-a632-7df878f9766a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f53086ba24643d5b406692708ff8c70', 'request_id': 'req-b4f3f1d2-b6eb-4f07-a632-7df878f9766a'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:54.220047 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-b4f3f1d2-b6eb-4f07-a632-7df878f9766a admin admin] The token cache is being invalidate because project 1f53086ba24643d5b406692708ff8c70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:49:54.220338 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-b4f3f1d2-b6eb-4f07-a632-7df878f9766a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:54.221014 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-b4f3f1d2-b6eb-4f07-a632-7df878f9766a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:54.222594 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 997/1991] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:49:54 2022] DELETE /identity/v3/projects/1f53086ba24643d5b406692708ff8c70 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 14:49:54.227753 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b71a2-e36c-4e43-97db-2f5758f3c7ec None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:54.228252 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b71a2-e36c-4e43-97db-2f5758f3c7ec None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:54.228348 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b71a2-e36c-4e43-97db-2f5758f3c7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:54.247371 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-d22b71a2-e36c-4e43-97db-2f5758f3c7ec None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:49:54.248602 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-d22b71a2-e36c-4e43-97db-2f5758f3c7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:54.313217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 995/1992] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:49:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:49:54.324386 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c4a60-b326-4c2f-83be-fac8df15a320 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:54.324824 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c4a60-b326-4c2f-83be-fac8df15a320 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:54.325343 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c4a60-b326-4c2f-83be-fac8df15a320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:54.346327 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-db2c4a60-b326-4c2f-83be-fac8df15a320 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:49:54.347644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-db2c4a60-b326-4c2f-83be-fac8df15a320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:54.357114 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 998/1993] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:49:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:49:54.361558 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b71a2-e36c-4e43-97db-2f5758f3c7ec None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:54.371721 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b71a2-e36c-4e43-97db-2f5758f3c7ec None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:54.376560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cae7dd-6728-4547-9683-cd963082dc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:54.377240 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cae7dd-6728-4547-9683-cd963082dc81 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:54.377520 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cae7dd-6728-4547-9683-cd963082dc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:54.377796 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cae7dd-6728-4547-9683-cd963082dc81 admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:54.378529 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cae7dd-6728-4547-9683-cd963082dc81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:49:54.378836 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cae7dd-6728-4547-9683-cd963082dc81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:54.380499 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cae7dd-6728-4547-9683-cd963082dc81 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:54.396653 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 996/1994] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:49:54.397886 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c4a60-b326-4c2f-83be-fac8df15a320 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:54.408387 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c4a60-b326-4c2f-83be-fac8df15a320 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:54.413372 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d448f-856c-44ba-9493-dc57e5bce0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:54.414053 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d448f-856c-44ba-9493-dc57e5bce0cc admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:54.414342 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d448f-856c-44ba-9493-dc57e5bce0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:54.414622 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d448f-856c-44ba-9493-dc57e5bce0cc admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:54.415372 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d448f-856c-44ba-9493-dc57e5bce0cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:54.417313 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d448f-856c-44ba-9493-dc57e5bce0cc admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:54.454645 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 999/1995] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:49:54 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:49:54.458903 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cae7dd-6728-4547-9683-cd963082dc81 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:54.469597 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cae7dd-6728-4547-9683-cd963082dc81 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:54.536403 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eabe48f-a744-40c1-91c2-e88759d8e6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:54.537095 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eabe48f-a744-40c1-91c2-e88759d8e6e8 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:54.537386 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eabe48f-a744-40c1-91c2-e88759d8e6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:54.537660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eabe48f-a744-40c1-91c2-e88759d8e6e8 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:54.538381 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eabe48f-a744-40c1-91c2-e88759d8e6e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:54.540355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eabe48f-a744-40c1-91c2-e88759d8e6e8 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:54.561301 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 997/1996] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:49:54 2022] POST /identity/v3/users => generated 285 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:49:54.565614 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d448f-856c-44ba-9493-dc57e5bce0cc admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:54.576224 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d448f-856c-44ba-9493-dc57e5bce0cc admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:54.581053 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3433aee-b625-4c76-9e82-16cd7a0af5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:54.581733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3433aee-b625-4c76-9e82-16cd7a0af5f5 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:54.582024 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3433aee-b625-4c76-9e82-16cd7a0af5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:54.582280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3433aee-b625-4c76-9e82-16cd7a0af5f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:54.583024 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3433aee-b625-4c76-9e82-16cd7a0af5f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:54.584680 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3433aee-b625-4c76-9e82-16cd7a0af5f5 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:54.596067 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1000/1997] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 14:49:54.600835 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eabe48f-a744-40c1-91c2-e88759d8e6e8 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:54.611854 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eabe48f-a744-40c1-91c2-e88759d8e6e8 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:54.617666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff9a31f-61fd-45f2-af43-84a7accf173e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:54.618360 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9a31f-61fd-45f2-af43-84a7accf173e admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:54.618647 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9a31f-61fd-45f2-af43-84a7accf173e admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:54.618924 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9a31f-61fd-45f2-af43-84a7accf173e admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:54.619720 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff9a31f-61fd-45f2-af43-84a7accf173e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:54.621351 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff9a31f-61fd-45f2-af43-84a7accf173e admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:54.632569 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 998/1998] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:49:54.637261 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3433aee-b625-4c76-9e82-16cd7a0af5f5 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:54.649080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3433aee-b625-4c76-9e82-16cd7a0af5f5 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:54.654012 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388386ad-7e35-41a7-b749-810f41b9d211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:54.655263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388386ad-7e35-41a7-b749-810f41b9d211 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:54.655552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388386ad-7e35-41a7-b749-810f41b9d211 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:54.655834 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388386ad-7e35-41a7-b749-810f41b9d211 admin admin] PATH_INFO: `/v3/projects/3c863ad683c74f7c9cbc253e416020f3/users/4a02cc1308c4492abac43e05e066661d/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:54.671820 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388386ad-7e35-41a7-b749-810f41b9d211 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c863ad683c74f7c9cbc253e416020f3, user_id=4a02cc1308c4492abac43e05e066661d, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:54.674502 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388386ad-7e35-41a7-b749-810f41b9d211 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:54.689114 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1001/1999] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:49:54 2022] PUT /identity/v3/projects/3c863ad683c74f7c9cbc253e416020f3/users/4a02cc1308c4492abac43e05e066661d/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 14:49:54.776158 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998e03a-62ff-41cf-b734-38271b0ac095 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:54.776440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998e03a-62ff-41cf-b734-38271b0ac095 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:54.776712 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998e03a-62ff-41cf-b734-38271b0ac095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:54.807504 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-f998e03a-62ff-41cf-b734-38271b0ac095 None None] MFA Rules not processed for user `4a02cc1308c4492abac43e05e066661d`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:49:54.808735 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-f998e03a-62ff-41cf-b734-38271b0ac095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:54.822390 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:54.881872 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 999/2000] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:49:54 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:49:54.893355 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388386ad-7e35-41a7-b749-810f41b9d211 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:54.895282 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-388386ad-7e35-41a7-b749-810f41b9d211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:54.965564 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388386ad-7e35-41a7-b749-810f41b9d211 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:54.969219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1370fd4-1b1a-43bc-bc41-050a7f4fac91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:54.969907 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1370fd4-1b1a-43bc-bc41-050a7f4fac91 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:54.970188 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1370fd4-1b1a-43bc-bc41-050a7f4fac91 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:54.970461 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1370fd4-1b1a-43bc-bc41-050a7f4fac91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:54.980001 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1370fd4-1b1a-43bc-bc41-050a7f4fac91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:54.981578 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1370fd4-1b1a-43bc-bc41-050a7f4fac91 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:54.991493 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1002/2001] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:49:54 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:49:55.017095 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998e03a-62ff-41cf-b734-38271b0ac095 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:55.103511 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998e03a-62ff-41cf-b734-38271b0ac095 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:55.113731 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d33eb-cbe2-438a-a27a-1be445edfc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:55.114677 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d33eb-cbe2-438a-a27a-1be445edfc7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:55.115079 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d33eb-cbe2-438a-a27a-1be445edfc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:55.115408 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d33eb-cbe2-438a-a27a-1be445edfc7f admin admin] PATH_INFO: `/v3/users/4a02cc1308c4492abac43e05e066661d` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:55.116789 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d33eb-cbe2-438a-a27a-1be445edfc7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a02cc1308c4492abac43e05e066661d)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:55.119033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d33eb-cbe2-438a-a27a-1be445edfc7f admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:55.170221 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-e84d33eb-cbe2-438a-a27a-1be445edfc7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a02cc1308c4492abac43e05e066661d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:55.183027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-e84d33eb-cbe2-438a-a27a-1be445edfc7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a02cc1308c4492abac43e05e066661d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:55.183787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-e84d33eb-cbe2-438a-a27a-1be445edfc7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a02cc1308c4492abac43e05e066661d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:55.184427 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-e84d33eb-cbe2-438a-a27a-1be445edfc7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a02cc1308c4492abac43e05e066661d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:55.191488 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-e84d33eb-cbe2-438a-a27a-1be445edfc7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a02cc1308c4492abac43e05e066661d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:55.205903 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1000/2002] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:49:55 2022] DELETE /identity/v3/users/4a02cc1308c4492abac43e05e066661d => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:49:55.210659 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed61454-9bf0-43f5-870c-409ed1ca0127 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:55.210976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed61454-9bf0-43f5-870c-409ed1ca0127 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:55.211256 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed61454-9bf0-43f5-870c-409ed1ca0127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:55.228827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-9ed61454-9bf0-43f5-870c-409ed1ca0127 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:49:55.230055 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-9ed61454-9bf0-43f5-870c-409ed1ca0127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:55.273069 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:55.296599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1003/2003] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:49:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:49:55.306866 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d33eb-cbe2-438a-a27a-1be445edfc7f admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:55.308456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e84d33eb-cbe2-438a-a27a-1be445edfc7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:55.353342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:55.373287 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d33eb-cbe2-438a-a27a-1be445edfc7f admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:55.377214 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb429b-541f-4731-86cc-18455e3e2322 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:55.377867 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb429b-541f-4731-86cc-18455e3e2322 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:55.378146 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb429b-541f-4731-86cc-18455e3e2322 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:55.378416 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb429b-541f-4731-86cc-18455e3e2322 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:55.388763 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb429b-541f-4731-86cc-18455e3e2322 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:55.390205 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb429b-541f-4731-86cc-18455e3e2322 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:55.400375 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1001/2004] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:49:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:49:55.789714 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed61454-9bf0-43f5-870c-409ed1ca0127 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:55.791520 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-9ed61454-9bf0-43f5-870c-409ed1ca0127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:55.812501 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed61454-9bf0-43f5-870c-409ed1ca0127 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:55.820447 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a21c03-17e8-4d8a-b8dc-000842831a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:55.821829 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a21c03-17e8-4d8a-b8dc-000842831a97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:55.822259 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a21c03-17e8-4d8a-b8dc-000842831a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:55.822696 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a21c03-17e8-4d8a-b8dc-000842831a97 admin admin] PATH_INFO: `/v3/projects/3c863ad683c74f7c9cbc253e416020f3` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:55.824158 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a21c03-17e8-4d8a-b8dc-000842831a97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c863ad683c74f7c9cbc253e416020f3)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:55.826081 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a21c03-17e8-4d8a-b8dc-000842831a97 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:55.881432 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-95a21c03-17e8-4d8a-b8dc-000842831a97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c863ad683c74f7c9cbc253e416020f3', 'request_id': 'req-95a21c03-17e8-4d8a-b8dc-000842831a97'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:55.888610 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-95a21c03-17e8-4d8a-b8dc-000842831a97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c863ad683c74f7c9cbc253e416020f3', 'request_id': 'req-95a21c03-17e8-4d8a-b8dc-000842831a97'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:55.899362 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-95a21c03-17e8-4d8a-b8dc-000842831a97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c863ad683c74f7c9cbc253e416020f3', 'request_id': 'req-95a21c03-17e8-4d8a-b8dc-000842831a97'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:55.911527 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-95a21c03-17e8-4d8a-b8dc-000842831a97 admin admin] The token cache is being invalidate because project 3c863ad683c74f7c9cbc253e416020f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:49:55.911846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-95a21c03-17e8-4d8a-b8dc-000842831a97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:55.912503 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-95a21c03-17e8-4d8a-b8dc-000842831a97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:49:55.914073 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1004/2005] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:49:55 2022] DELETE /identity/v3/projects/3c863ad683c74f7c9cbc253e416020f3 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:49:55.919511 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef186d-5b5d-4d45-a3ad-63d32ce7132d None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:55.919796 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef186d-5b5d-4d45-a3ad-63d32ce7132d None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:55.920077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef186d-5b5d-4d45-a3ad-63d32ce7132d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:55.938335 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-1aef186d-5b5d-4d45-a3ad-63d32ce7132d None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:49:55.939586 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-1aef186d-5b5d-4d45-a3ad-63d32ce7132d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:56.003218 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1002/2006] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:49:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:49:56.010842 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557fabcf-b0d0-42b7-9771-ece97765040a None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:56.011157 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557fabcf-b0d0-42b7-9771-ece97765040a None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:56.011443 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557fabcf-b0d0-42b7-9771-ece97765040a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:56.031804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-557fabcf-b0d0-42b7-9771-ece97765040a None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:49:56.033001 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-557fabcf-b0d0-42b7-9771-ece97765040a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:56.041549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1005/2007] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:49:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:49:56.046556 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef186d-5b5d-4d45-a3ad-63d32ce7132d None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:56.057533 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef186d-5b5d-4d45-a3ad-63d32ce7132d None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:56.063593 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdd94e-6a8a-4e98-8699-ce1cc028a965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:56.064256 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdd94e-6a8a-4e98-8699-ce1cc028a965 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:56.064542 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdd94e-6a8a-4e98-8699-ce1cc028a965 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:56.064834 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdd94e-6a8a-4e98-8699-ce1cc028a965 admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:56.065594 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccdd94e-6a8a-4e98-8699-ce1cc028a965 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:49:56.065902 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccdd94e-6a8a-4e98-8699-ce1cc028a965 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:56.067553 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccdd94e-6a8a-4e98-8699-ce1cc028a965 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:56.080844 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1003/2008] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:49:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:49:56.085447 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557fabcf-b0d0-42b7-9771-ece97765040a None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:56.096613 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557fabcf-b0d0-42b7-9771-ece97765040a None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:56.102506 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50e210-2cde-4e06-93ff-90566ccb3fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:56.103224 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50e210-2cde-4e06-93ff-90566ccb3fad admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:56.103513 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50e210-2cde-4e06-93ff-90566ccb3fad admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:56.103790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50e210-2cde-4e06-93ff-90566ccb3fad admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:56.104500 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe50e210-2cde-4e06-93ff-90566ccb3fad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:56.106486 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe50e210-2cde-4e06-93ff-90566ccb3fad admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:56.139034 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1006/2009] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:49:56 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:49:56.140457 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdd94e-6a8a-4e98-8699-ce1cc028a965 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:56.153683 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdd94e-6a8a-4e98-8699-ce1cc028a965 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:56.223787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f6f4c-63f5-4d91-b4d7-3c6ae70f9ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:56.224492 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f6f4c-63f5-4d91-b4d7-3c6ae70f9ba6 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:56.224780 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f6f4c-63f5-4d91-b4d7-3c6ae70f9ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:56.225062 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f6f4c-63f5-4d91-b4d7-3c6ae70f9ba6 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:56.225778 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f6f4c-63f5-4d91-b4d7-3c6ae70f9ba6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:56.227778 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f6f4c-63f5-4d91-b4d7-3c6ae70f9ba6 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:56.250528 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1004/2010] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:49:56 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:49:56.255195 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50e210-2cde-4e06-93ff-90566ccb3fad admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:56.266042 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50e210-2cde-4e06-93ff-90566ccb3fad admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:56.271135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a5528-e1c9-4a6b-bf50-9e74b183ee2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:56.271837 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a5528-e1c9-4a6b-bf50-9e74b183ee2e admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:56.272124 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a5528-e1c9-4a6b-bf50-9e74b183ee2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:56.272405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a5528-e1c9-4a6b-bf50-9e74b183ee2e admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:56.273150 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542a5528-e1c9-4a6b-bf50-9e74b183ee2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:56.274763 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542a5528-e1c9-4a6b-bf50-9e74b183ee2e admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:56.286813 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1007/2011] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:49:56.292107 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f6f4c-63f5-4d91-b4d7-3c6ae70f9ba6 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:56.309215 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f6f4c-63f5-4d91-b4d7-3c6ae70f9ba6 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:56.317461 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3632a6-b17a-446b-bad3-1bed6b3780f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:56.318583 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3632a6-b17a-446b-bad3-1bed6b3780f7 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:56.319128 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3632a6-b17a-446b-bad3-1bed6b3780f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:56.319641 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3632a6-b17a-446b-bad3-1bed6b3780f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:56.320903 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3632a6-b17a-446b-bad3-1bed6b3780f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:56.323846 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3632a6-b17a-446b-bad3-1bed6b3780f7 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:56.340708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1005/2012] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:49:56.345422 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a5528-e1c9-4a6b-bf50-9e74b183ee2e admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:56.355972 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a5528-e1c9-4a6b-bf50-9e74b183ee2e admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:56.361022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f9060-6a4a-4f1a-b99e-48f5624ff0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:56.361968 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f9060-6a4a-4f1a-b99e-48f5624ff0b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:56.362249 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f9060-6a4a-4f1a-b99e-48f5624ff0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:56.362507 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f9060-6a4a-4f1a-b99e-48f5624ff0b1 admin admin] PATH_INFO: `/v3/projects/50a5f7380c804eba98ab4a8ee65dc1d3/users/e1750ad6625e4cc0b430a81ed188e538/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:56.379415 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f9060-6a4a-4f1a-b99e-48f5624ff0b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50a5f7380c804eba98ab4a8ee65dc1d3, user_id=e1750ad6625e4cc0b430a81ed188e538, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:56.382274 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f9060-6a4a-4f1a-b99e-48f5624ff0b1 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:56.398782 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1008/2013] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:49:56 2022] PUT /identity/v3/projects/50a5f7380c804eba98ab4a8ee65dc1d3/users/e1750ad6625e4cc0b430a81ed188e538/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:49:56.409963 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac712d0-16e0-4ead-9d42-6e31dea7bd9b None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:56.410179 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac712d0-16e0-4ead-9d42-6e31dea7bd9b None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:56.410461 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac712d0-16e0-4ead-9d42-6e31dea7bd9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:56.439059 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-dac712d0-16e0-4ead-9d42-6e31dea7bd9b None None] MFA Rules not processed for user `e1750ad6625e4cc0b430a81ed188e538`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:49:56.440329 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-dac712d0-16e0-4ead-9d42-6e31dea7bd9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:56.473167 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:56.510558 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1006/2014] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:49:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:49:56.521536 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f9060-6a4a-4f1a-b99e-48f5624ff0b1 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:49:56.523626 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-f24f9060-6a4a-4f1a-b99e-48f5624ff0b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:56.588000 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f9060-6a4a-4f1a-b99e-48f5624ff0b1 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:49:56.594792 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbe687-f674-4361-9f15-8ddd36428316 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:49:56.595514 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbe687-f674-4361-9f15-8ddd36428316 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:56.595789 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbe687-f674-4361-9f15-8ddd36428316 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:56.596060 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbe687-f674-4361-9f15-8ddd36428316 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:56.605916 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbbe687-f674-4361-9f15-8ddd36428316 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:49:56.608261 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbbe687-f674-4361-9f15-8ddd36428316 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:49:56.618716 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1009/2015] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:49:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:49:59.045290 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2a24e-bc43-4a74-af8f-86e1f4e87a91 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:49:59.046058 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2a24e-bc43-4a74-af8f-86e1f4e87a91 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:49:59.046058 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2a24e-bc43-4a74-af8f-86e1f4e87a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:49:59.067134 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-66f2a24e-bc43-4a74-af8f-86e1f4e87a91 None None] MFA Rules not processed for user `e1750ad6625e4cc0b430a81ed188e538`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:49:59.068399 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-66f2a24e-bc43-4a74-af8f-86e1f4e87a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:49:59.079503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1007/2016] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:49:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:06.614098 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:09.061319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=15 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:17.473970 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbe687-f674-4361-9f15-8ddd36428316 service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:17.475602 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-fbbbe687-f674-4361-9f15-8ddd36428316 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:17.564968 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbe687-f674-4361-9f15-8ddd36428316 service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:17.572906 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20a561-28e5-4f3d-b187-27de9d671bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:17.574535 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20a561-28e5-4f3d-b187-27de9d671bda admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:17.575293 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20a561-28e5-4f3d-b187-27de9d671bda admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:17.575789 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20a561-28e5-4f3d-b187-27de9d671bda admin admin] PATH_INFO: `/v3/users/c010da5eaee94134b32ffe64c36861d5` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:17.577728 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e20a561-28e5-4f3d-b187-27de9d671bda admin admin] RBAC: Authorizing `identity:delete_user(user_id=c010da5eaee94134b32ffe64c36861d5)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:17.581502 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e20a561-28e5-4f3d-b187-27de9d671bda admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:17.647923 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-5e20a561-28e5-4f3d-b187-27de9d671bda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c010da5eaee94134b32ffe64c36861d5'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:17.673241 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-5e20a561-28e5-4f3d-b187-27de9d671bda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c010da5eaee94134b32ffe64c36861d5'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:17.674183 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-5e20a561-28e5-4f3d-b187-27de9d671bda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c010da5eaee94134b32ffe64c36861d5'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:17.675002 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-5e20a561-28e5-4f3d-b187-27de9d671bda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c010da5eaee94134b32ffe64c36861d5'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:17.682778 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-5e20a561-28e5-4f3d-b187-27de9d671bda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c010da5eaee94134b32ffe64c36861d5'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:17.697365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1010/2017] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:50:17 2022] DELETE /identity/v3/users/c010da5eaee94134b32ffe64c36861d5 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:50:17.701997 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1820cee-3956-426f-9acc-7865483e0b47 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:17.702907 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1820cee-3956-426f-9acc-7865483e0b47 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:17.703225 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1820cee-3956-426f-9acc-7865483e0b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:17.722979 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-d1820cee-3956-426f-9acc-7865483e0b47 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:17.724540 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-d1820cee-3956-426f-9acc-7865483e0b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:17.791477 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1008/2018] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:50:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:50:17.800039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20a561-28e5-4f3d-b187-27de9d671bda admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:17.802192 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-5e20a561-28e5-4f3d-b187-27de9d671bda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:17.868987 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20a561-28e5-4f3d-b187-27de9d671bda admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:17.884044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4bfdd-ca60-4325-8a7d-cc424b1e04f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:17.884729 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4bfdd-ca60-4325-8a7d-cc424b1e04f1 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:17.885017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4bfdd-ca60-4325-8a7d-cc424b1e04f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:17.885297 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4bfdd-ca60-4325-8a7d-cc424b1e04f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:17.898270 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d4bfdd-ca60-4325-8a7d-cc424b1e04f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:17.899866 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d4bfdd-ca60-4325-8a7d-cc424b1e04f1 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:17.919127 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1011/2019] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:50:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:50:18.436602 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1820cee-3956-426f-9acc-7865483e0b47 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:18.438648 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-d1820cee-3956-426f-9acc-7865483e0b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:18.453059 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1820cee-3956-426f-9acc-7865483e0b47 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:18.460388 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74863d7e-1700-4125-881c-eb1f91a496e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:18.461031 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74863d7e-1700-4125-881c-eb1f91a496e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:18.461415 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74863d7e-1700-4125-881c-eb1f91a496e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:18.462074 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74863d7e-1700-4125-881c-eb1f91a496e1 admin admin] PATH_INFO: `/v3/projects/a63e6efde0ec4d8fac265d68dd0f21a0` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:18.463625 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74863d7e-1700-4125-881c-eb1f91a496e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a63e6efde0ec4d8fac265d68dd0f21a0)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:18.465707 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74863d7e-1700-4125-881c-eb1f91a496e1 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:18.536057 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-74863d7e-1700-4125-881c-eb1f91a496e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a63e6efde0ec4d8fac265d68dd0f21a0', 'request_id': 'req-74863d7e-1700-4125-881c-eb1f91a496e1'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:18.548297 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-74863d7e-1700-4125-881c-eb1f91a496e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a63e6efde0ec4d8fac265d68dd0f21a0', 'request_id': 'req-74863d7e-1700-4125-881c-eb1f91a496e1'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:18.556048 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-74863d7e-1700-4125-881c-eb1f91a496e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a63e6efde0ec4d8fac265d68dd0f21a0', 'request_id': 'req-74863d7e-1700-4125-881c-eb1f91a496e1'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:18.572342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-74863d7e-1700-4125-881c-eb1f91a496e1 admin admin] The token cache is being invalidate because project a63e6efde0ec4d8fac265d68dd0f21a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:50:18.572819 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-74863d7e-1700-4125-881c-eb1f91a496e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:18.573573 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-74863d7e-1700-4125-881c-eb1f91a496e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:18.578181 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1009/2020] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:50:18 2022] DELETE /identity/v3/projects/a63e6efde0ec4d8fac265d68dd0f21a0 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:50:18.580132 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4bfdd-ca60-4325-8a7d-cc424b1e04f1 service neutron] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:18.599514 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4bfdd-ca60-4325-8a7d-cc424b1e04f1 service neutron] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:18.630071 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:18.713096 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f574e3-0307-464e-b0b0-bf217eadad6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:18.714092 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f574e3-0307-464e-b0b0-bf217eadad6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:18.714269 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f574e3-0307-464e-b0b0-bf217eadad6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:18.714553 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f574e3-0307-464e-b0b0-bf217eadad6b admin admin] PATH_INFO: `/v3/users/3479314733f44debbcc54f5c56811e60` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:18.715970 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f574e3-0307-464e-b0b0-bf217eadad6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3479314733f44debbcc54f5c56811e60)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:18.721706 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f574e3-0307-464e-b0b0-bf217eadad6b admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:18.784366 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-38f574e3-0307-464e-b0b0-bf217eadad6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3479314733f44debbcc54f5c56811e60'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:18.798088 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-38f574e3-0307-464e-b0b0-bf217eadad6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3479314733f44debbcc54f5c56811e60'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:18.798914 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-38f574e3-0307-464e-b0b0-bf217eadad6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3479314733f44debbcc54f5c56811e60'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:18.799784 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-38f574e3-0307-464e-b0b0-bf217eadad6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3479314733f44debbcc54f5c56811e60'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:18.816440 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-38f574e3-0307-464e-b0b0-bf217eadad6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3479314733f44debbcc54f5c56811e60'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:18.839129 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1012/2021] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:50:18 2022] DELETE /identity/v3/users/3479314733f44debbcc54f5c56811e60 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:50:19.029634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cc68b-3637-46d5-a947-23f999c6a600 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:19.030191 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cc68b-3637-46d5-a947-23f999c6a600 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:19.030191 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cc68b-3637-46d5-a947-23f999c6a600 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:19.031595 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1010/2022] 10.222.0.15 () {54 vars in 921 bytes} [Wed Jul 20 14:50:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:50:19.042978 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e304e-960c-4012-a305-0b8c242bb301 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:19.043409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e304e-960c-4012-a305-0b8c242bb301 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:19.043767 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e304e-960c-4012-a305-0b8c242bb301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:19.065926 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-f56e304e-960c-4012-a305-0b8c242bb301 None None] MFA Rules not processed for user `cb857f32bbc04d82bfa681bd5659765f`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:19.067346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-f56e304e-960c-4012-a305-0b8c242bb301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:19.134836 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691cc68b-3637-46d5-a947-23f999c6a600 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:19.136734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-691cc68b-3637-46d5-a947-23f999c6a600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:19.168649 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1013/2023] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Jul 20 14:50:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:19.170484 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e304e-960c-4012-a305-0b8c242bb301 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:19.172381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-f56e304e-960c-4012-a305-0b8c242bb301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:19.226015 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:19.248326 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e304e-960c-4012-a305-0b8c242bb301 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:19.250870 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691cc68b-3637-46d5-a947-23f999c6a600 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:19.252754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81815010-abb4-4be4-b1cb-a265b162c939 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:19.253435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81815010-abb4-4be4-b1cb-a265b162c939 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:19.253813 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81815010-abb4-4be4-b1cb-a265b162c939 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:19.254044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81815010-abb4-4be4-b1cb-a265b162c939 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:19.256181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-81815010-abb4-4be4-b1cb-a265b162c939 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:19.256560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09797e-d821-41f9-8c27-443ace3cddb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:19.257494 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09797e-d821-41f9-8c27-443ace3cddb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:19.257961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09797e-d821-41f9-8c27-443ace3cddb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:19.258286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09797e-d821-41f9-8c27-443ace3cddb1 admin admin] PATH_INFO: `/v3/users/5373c0299ea64c9f872e3bc04bfb0b74` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:19.259508 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd09797e-d821-41f9-8c27-443ace3cddb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5373c0299ea64c9f872e3bc04bfb0b74)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:19.261555 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd09797e-d821-41f9-8c27-443ace3cddb1 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:19.291962 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:19.316957 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-dd09797e-d821-41f9-8c27-443ace3cddb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5373c0299ea64c9f872e3bc04bfb0b74'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:19.331355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-dd09797e-d821-41f9-8c27-443ace3cddb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5373c0299ea64c9f872e3bc04bfb0b74'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:19.332591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-dd09797e-d821-41f9-8c27-443ace3cddb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5373c0299ea64c9f872e3bc04bfb0b74'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:19.333678 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-dd09797e-d821-41f9-8c27-443ace3cddb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5373c0299ea64c9f872e3bc04bfb0b74'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:19.336084 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81815010-abb4-4be4-b1cb-a265b162c939 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:19.338027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81815010-abb4-4be4-b1cb-a265b162c939 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:19.349087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-dd09797e-d821-41f9-8c27-443ace3cddb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5373c0299ea64c9f872e3bc04bfb0b74'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:19.349541 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1014/2024] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:50:19 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:50:19.350465 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81815010-abb4-4be4-b1cb-a265b162c939 service neutron] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:19.352113 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-81815010-abb4-4be4-b1cb-a265b162c939 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:19.368357 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1011/2025] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:50:19 2022] DELETE /identity/v3/users/5373c0299ea64c9f872e3bc04bfb0b74 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 14:50:19.368516 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81815010-abb4-4be4-b1cb-a265b162c939 service neutron] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:19.377094 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09797e-d821-41f9-8c27-443ace3cddb1 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:19.378884 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-dd09797e-d821-41f9-8c27-443ace3cddb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:19.465776 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b6adc5-e01a-48aa-ae7f-a5403cfd6a79 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb857f32bbc04d82bfa681bd5659765f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:19.469987 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6adc5-e01a-48aa-ae7f-a5403cfd6a79 service placement] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:19.470367 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6adc5-e01a-48aa-ae7f-a5403cfd6a79 service placement] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:19.470695 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6adc5-e01a-48aa-ae7f-a5403cfd6a79 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:19.473604 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-e8b6adc5-e01a-48aa-ae7f-a5403cfd6a79 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:19.479450 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09797e-d821-41f9-8c27-443ace3cddb1 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:19.485321 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071dd7a0-291c-4a0c-9dcd-102e4cebb949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:19.486445 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071dd7a0-291c-4a0c-9dcd-102e4cebb949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:19.487556 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071dd7a0-291c-4a0c-9dcd-102e4cebb949 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:19.488001 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071dd7a0-291c-4a0c-9dcd-102e4cebb949 admin admin] PATH_INFO: `/v3/projects/39fe469725664b1ab6a4c6dd63d1a0c8` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:19.489740 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071dd7a0-291c-4a0c-9dcd-102e4cebb949 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39fe469725664b1ab6a4c6dd63d1a0c8)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:19.492702 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071dd7a0-291c-4a0c-9dcd-102e4cebb949 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:19.495928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b6adc5-e01a-48aa-ae7f-a5403cfd6a79 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:19.497920 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b6adc5-e01a-48aa-ae7f-a5403cfd6a79 service placement] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:19.522527 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1015/2026] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:50:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:50:19.591217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-071dd7a0-291c-4a0c-9dcd-102e4cebb949 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39fe469725664b1ab6a4c6dd63d1a0c8', 'request_id': 'req-071dd7a0-291c-4a0c-9dcd-102e4cebb949'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:19.596395 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b6adc5-e01a-48aa-ae7f-a5403cfd6a79 service placement] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:19.598326 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-e8b6adc5-e01a-48aa-ae7f-a5403cfd6a79 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:19.601766 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-071dd7a0-291c-4a0c-9dcd-102e4cebb949 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39fe469725664b1ab6a4c6dd63d1a0c8', 'request_id': 'req-071dd7a0-291c-4a0c-9dcd-102e4cebb949'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:19.608985 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-071dd7a0-291c-4a0c-9dcd-102e4cebb949 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39fe469725664b1ab6a4c6dd63d1a0c8', 'request_id': 'req-071dd7a0-291c-4a0c-9dcd-102e4cebb949'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:19.618335 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b6adc5-e01a-48aa-ae7f-a5403cfd6a79 service placement] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:19.625779 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-071dd7a0-291c-4a0c-9dcd-102e4cebb949 admin admin] The token cache is being invalidate because project 39fe469725664b1ab6a4c6dd63d1a0c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:50:19.626280 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-071dd7a0-291c-4a0c-9dcd-102e4cebb949 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:19.627070 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-071dd7a0-291c-4a0c-9dcd-102e4cebb949 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:19.629377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1012/2027] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:50:19 2022] DELETE /identity/v3/projects/39fe469725664b1ab6a4c6dd63d1a0c8 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 14:50:19.634163 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ccd1f-2ff8-4c19-92d9-9b78d6f9a72a None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:19.634458 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ccd1f-2ff8-4c19-92d9-9b78d6f9a72a None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:19.634739 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ccd1f-2ff8-4c19-92d9-9b78d6f9a72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:19.660705 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-732ccd1f-2ff8-4c19-92d9-9b78d6f9a72a None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:19.662555 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-732ccd1f-2ff8-4c19-92d9-9b78d6f9a72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:19.703917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:19.712636 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc915aa7-fac7-4377-a7d9-cc2f7e46abeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:19.713688 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc915aa7-fac7-4377-a7d9-cc2f7e46abeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:19.714186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc915aa7-fac7-4377-a7d9-cc2f7e46abeb admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:19.714562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc915aa7-fac7-4377-a7d9-cc2f7e46abeb admin admin] PATH_INFO: `/v3/projects/3d28cd25ee0c476e8ab3b24d7f2f9ab9` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:19.716109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc915aa7-fac7-4377-a7d9-cc2f7e46abeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d28cd25ee0c476e8ab3b24d7f2f9ab9)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:19.718756 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc915aa7-fac7-4377-a7d9-cc2f7e46abeb admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:19.751181 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1013/2028] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:50:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:19.760137 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f297cd-dfa0-41d6-b98a-08ce7b72a994 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:19.760433 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f297cd-dfa0-41d6-b98a-08ce7b72a994 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:19.760744 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f297cd-dfa0-41d6-b98a-08ce7b72a994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:19.762645 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:19.802831 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-41f297cd-dfa0-41d6-b98a-08ce7b72a994 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:19.804827 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-41f297cd-dfa0-41d6-b98a-08ce7b72a994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:19.805754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-dc915aa7-fac7-4377-a7d9-cc2f7e46abeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'request_id': 'req-dc915aa7-fac7-4377-a7d9-cc2f7e46abeb'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:19.811954 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-dc915aa7-fac7-4377-a7d9-cc2f7e46abeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'request_id': 'req-dc915aa7-fac7-4377-a7d9-cc2f7e46abeb'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:19.821665 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-dc915aa7-fac7-4377-a7d9-cc2f7e46abeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'request_id': 'req-dc915aa7-fac7-4377-a7d9-cc2f7e46abeb'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:19.837948 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-dc915aa7-fac7-4377-a7d9-cc2f7e46abeb admin admin] The token cache is being invalidate because project 3d28cd25ee0c476e8ab3b24d7f2f9ab9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:50:19.838460 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-dc915aa7-fac7-4377-a7d9-cc2f7e46abeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:19.839381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-dc915aa7-fac7-4377-a7d9-cc2f7e46abeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:19.841391 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1016/2029] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:50:19 2022] DELETE /identity/v3/projects/3d28cd25ee0c476e8ab3b24d7f2f9ab9 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:50:19.846387 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993cc4f-0944-4ef5-8536-aa7da0e8bcd1 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:19.846669 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993cc4f-0944-4ef5-8536-aa7da0e8bcd1 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:19.846943 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993cc4f-0944-4ef5-8536-aa7da0e8bcd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:19.873208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-d993cc4f-0944-4ef5-8536-aa7da0e8bcd1 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:19.874447 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-d993cc4f-0944-4ef5-8536-aa7da0e8bcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:19.900039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1014/2030] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:50:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:19.904533 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f297cd-dfa0-41d6-b98a-08ce7b72a994 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:19.916501 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f297cd-dfa0-41d6-b98a-08ce7b72a994 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:19.922305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d889df7f-5a5e-45ba-a020-63672a4e9bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:19.923454 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889df7f-5a5e-45ba-a020-63672a4e9bcf admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:19.923702 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889df7f-5a5e-45ba-a020-63672a4e9bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:19.923982 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889df7f-5a5e-45ba-a020-63672a4e9bcf admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:19.924820 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d889df7f-5a5e-45ba-a020-63672a4e9bcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:50:19.925133 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d889df7f-5a5e-45ba-a020-63672a4e9bcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:19.927037 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d889df7f-5a5e-45ba-a020-63672a4e9bcf admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:19.944437 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1015/2031] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:50:19.949940 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d889df7f-5a5e-45ba-a020-63672a4e9bcf admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:19.964313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d889df7f-5a5e-45ba-a020-63672a4e9bcf admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:19.971063 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd9544-7827-4aa5-8163-76b3b72c7d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:19.971839 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd9544-7827-4aa5-8163-76b3b72c7d45 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:19.972188 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd9544-7827-4aa5-8163-76b3b72c7d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:19.972623 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd9544-7827-4aa5-8163-76b3b72c7d45 admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:19.973457 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd9544-7827-4aa5-8163-76b3b72c7d45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:19.976073 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd9544-7827-4aa5-8163-76b3b72c7d45 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:19.976198 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1017/2032] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:50:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:19.990413 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ded4d1-27fe-429d-bbcd-e92703f71746 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:19.990705 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ded4d1-27fe-429d-bbcd-e92703f71746 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:19.990990 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ded4d1-27fe-429d-bbcd-e92703f71746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:20.016942 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1016/2033] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:50:19 2022] POST /identity/v3/projects => generated 371 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:50:20.021423 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd9544-7827-4aa5-8163-76b3b72c7d45 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:20.025056 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-98ded4d1-27fe-429d-bbcd-e92703f71746 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:20.026383 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-98ded4d1-27fe-429d-bbcd-e92703f71746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:20.098065 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1018/2034] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:50:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:20.102018 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ded4d1-27fe-429d-bbcd-e92703f71746 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:20.105339 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd9544-7827-4aa5-8163-76b3b72c7d45 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:20.111400 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feaeaf6-68f6-4a05-8c82-2eab0c4cfc68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:20.113127 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ded4d1-27fe-429d-bbcd-e92703f71746 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:20.113460 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feaeaf6-68f6-4a05-8c82-2eab0c4cfc68 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:20.113817 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feaeaf6-68f6-4a05-8c82-2eab0c4cfc68 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:20.115240 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feaeaf6-68f6-4a05-8c82-2eab0c4cfc68 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:20.116095 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feaeaf6-68f6-4a05-8c82-2eab0c4cfc68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:20.118097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feaeaf6-68f6-4a05-8c82-2eab0c4cfc68 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:20.119798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a48c25-e088-4c0e-9218-b84ee16fd278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:20.120756 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a48c25-e088-4c0e-9218-b84ee16fd278 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:20.121116 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a48c25-e088-4c0e-9218-b84ee16fd278 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:20.122030 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a48c25-e088-4c0e-9218-b84ee16fd278 admin admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:20.122785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a48c25-e088-4c0e-9218-b84ee16fd278 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:50:20.123228 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a48c25-e088-4c0e-9218-b84ee16fd278 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:20.126836 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a48c25-e088-4c0e-9218-b84ee16fd278 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:20.145124 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1017/2035] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:50:20 2022] POST /identity/v3/users => generated 286 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:50:20.147386 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1019/2036] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:50:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:50:20.148512 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feaeaf6-68f6-4a05-8c82-2eab0c4cfc68 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:20.148639 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a48c25-e088-4c0e-9218-b84ee16fd278 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:20.162773 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a48c25-e088-4c0e-9218-b84ee16fd278 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:20.163328 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feaeaf6-68f6-4a05-8c82-2eab0c4cfc68 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:20.170740 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faebd8c4-1563-4bf9-aee1-5cb6b3477f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:20.172113 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06562d0c-de0a-48c1-92e5-b9d898b0c45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:20.172341 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faebd8c4-1563-4bf9-aee1-5cb6b3477f0c admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:20.172791 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faebd8c4-1563-4bf9-aee1-5cb6b3477f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:20.172893 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06562d0c-de0a-48c1-92e5-b9d898b0c45b admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:20.173269 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06562d0c-de0a-48c1-92e5-b9d898b0c45b admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:20.173530 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faebd8c4-1563-4bf9-aee1-5cb6b3477f0c admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:20.173636 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06562d0c-de0a-48c1-92e5-b9d898b0c45b admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:20.174610 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06562d0c-de0a-48c1-92e5-b9d898b0c45b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:20.176923 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06562d0c-de0a-48c1-92e5-b9d898b0c45b admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:20.177210 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faebd8c4-1563-4bf9-aee1-5cb6b3477f0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:20.179261 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faebd8c4-1563-4bf9-aee1-5cb6b3477f0c admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:20.194129 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1020/2037] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:50:20 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:50:20.195348 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faebd8c4-1563-4bf9-aee1-5cb6b3477f0c admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:20.207036 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faebd8c4-1563-4bf9-aee1-5cb6b3477f0c admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:20.213043 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce609c9-89e1-439b-8218-d2b8e61a016b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:20.214772 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce609c9-89e1-439b-8218-d2b8e61a016b admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:20.215190 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce609c9-89e1-439b-8218-d2b8e61a016b admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:20.215561 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce609c9-89e1-439b-8218-d2b8e61a016b admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:20.217092 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce609c9-89e1-439b-8218-d2b8e61a016b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:20.219348 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce609c9-89e1-439b-8218-d2b8e61a016b admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:20.223406 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1018/2038] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:50:20 2022] POST /identity/v3/projects => generated 385 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:50:20.228039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06562d0c-de0a-48c1-92e5-b9d898b0c45b admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:20.238167 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1021/2039] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:50:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:50:20.239435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce609c9-89e1-439b-8218-d2b8e61a016b admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:20.243189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:20.261395 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce609c9-89e1-439b-8218-d2b8e61a016b admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:20.282253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:20.331040 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06562d0c-de0a-48c1-92e5-b9d898b0c45b admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:20.347249 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa594529-9e2c-4bd6-b51f-fd380c0ff7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:20.348460 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa594529-9e2c-4bd6-b51f-fd380c0ff7e2 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:20.349086 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa594529-9e2c-4bd6-b51f-fd380c0ff7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:20.349543 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa594529-9e2c-4bd6-b51f-fd380c0ff7e2 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:20.351470 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa594529-9e2c-4bd6-b51f-fd380c0ff7e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:20.355258 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa594529-9e2c-4bd6-b51f-fd380c0ff7e2 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:20.365247 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0861ce-fc1d-4967-87ea-6fcf40cd0dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:20.366317 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0861ce-fc1d-4967-87ea-6fcf40cd0dff admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:20.366668 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0861ce-fc1d-4967-87ea-6fcf40cd0dff admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:20.367397 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0861ce-fc1d-4967-87ea-6fcf40cd0dff admin admin] PATH_INFO: `/v3/projects/3c44d203fb1a490f8b81af15dc9e34b5/users/ef3db5a2898a4ef2b8ca559163fcb1b8/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:20.390637 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1019/2040] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:50:20 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:50:20.394936 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa594529-9e2c-4bd6-b51f-fd380c0ff7e2 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:20.398588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0861ce-fc1d-4967-87ea-6fcf40cd0dff admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c44d203fb1a490f8b81af15dc9e34b5, user_id=ef3db5a2898a4ef2b8ca559163fcb1b8, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:20.401301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0861ce-fc1d-4967-87ea-6fcf40cd0dff admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:20.417466 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa594529-9e2c-4bd6-b51f-fd380c0ff7e2 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:20.426021 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1022/2041] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:50:20 2022] PUT /identity/v3/projects/3c44d203fb1a490f8b81af15dc9e34b5/users/ef3db5a2898a4ef2b8ca559163fcb1b8/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:50:20.434188 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb55ca-b4a9-4cf0-9916-acd35c1cc6c1 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:20.434361 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb55ca-b4a9-4cf0-9916-acd35c1cc6c1 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:20.435420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb55ca-b4a9-4cf0-9916-acd35c1cc6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:20.474872 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-2edb55ca-b4a9-4cf0-9916-acd35c1cc6c1 None None] MFA Rules not processed for user `ef3db5a2898a4ef2b8ca559163fcb1b8`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:20.479122 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-2edb55ca-b4a9-4cf0-9916-acd35c1cc6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:20.547251 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aa07a0-8e64-4a43-8799-fa5ca9022d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:20.549730 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa07a0-8e64-4a43-8799-fa5ca9022d8c admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:20.551354 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa07a0-8e64-4a43-8799-fa5ca9022d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:20.551712 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa07a0-8e64-4a43-8799-fa5ca9022d8c admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:20.553425 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aa07a0-8e64-4a43-8799-fa5ca9022d8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:20.557149 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aa07a0-8e64-4a43-8799-fa5ca9022d8c admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:20.575083 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1020/2042] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:50:20 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 14:50:20.577115 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aa07a0-8e64-4a43-8799-fa5ca9022d8c admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:20.599345 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aa07a0-8e64-4a43-8799-fa5ca9022d8c admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:20.617409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f12f6-d1ef-419c-9099-0f76a0311527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:20.618171 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f12f6-d1ef-419c-9099-0f76a0311527 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:20.618468 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f12f6-d1ef-419c-9099-0f76a0311527 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:20.622708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f12f6-d1ef-419c-9099-0f76a0311527 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:20.623605 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419f12f6-d1ef-419c-9099-0f76a0311527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:20.625726 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1023/2043] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:50:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:20.631735 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419f12f6-d1ef-419c-9099-0f76a0311527 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:20.637930 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb55ca-b4a9-4cf0-9916-acd35c1cc6c1 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:20.640745 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-2edb55ca-b4a9-4cf0-9916-acd35c1cc6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:20.663987 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1021/2044] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:50:20 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:50:20.665235 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f12f6-d1ef-419c-9099-0f76a0311527 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:20.686451 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f12f6-d1ef-419c-9099-0f76a0311527 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:20.695493 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a7f50-93a9-4b23-aa52-2d24276263f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:20.696489 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a7f50-93a9-4b23-aa52-2d24276263f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:20.697057 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a7f50-93a9-4b23-aa52-2d24276263f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:20.697354 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a7f50-93a9-4b23-aa52-2d24276263f5 admin admin] PATH_INFO: `/v3/projects/f0af7082b1c04ea687097e35e9050807/users/54f0a23979014886bfed56cf62a0fa70/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:20.720862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a7f50-93a9-4b23-aa52-2d24276263f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0af7082b1c04ea687097e35e9050807, user_id=54f0a23979014886bfed56cf62a0fa70, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:20.723848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a7f50-93a9-4b23-aa52-2d24276263f5 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:20.745284 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1022/2045] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:50:20 2022] PUT /identity/v3/projects/f0af7082b1c04ea687097e35e9050807/users/54f0a23979014886bfed56cf62a0fa70/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:50:20.749247 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb55ca-b4a9-4cf0-9916-acd35c1cc6c1 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:20.755564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757f7e3-ce9d-4078-a024-fda87122dd2e None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:20.755857 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757f7e3-ce9d-4078-a024-fda87122dd2e None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:20.756143 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757f7e3-ce9d-4078-a024-fda87122dd2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:20.789770 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-b757f7e3-ce9d-4078-a024-fda87122dd2e None None] MFA Rules not processed for user `54f0a23979014886bfed56cf62a0fa70`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:20.791040 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-b757f7e3-ce9d-4078-a024-fda87122dd2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:20.810853 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:20.833592 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f872ddb-5f79-4ceb-a993-b51e521eef7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:20.834298 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f872ddb-5f79-4ceb-a993-b51e521eef7d service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:20.834826 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f872ddb-5f79-4ceb-a993-b51e521eef7d service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:20.835217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f872ddb-5f79-4ceb-a993-b51e521eef7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:20.863894 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:20.916462 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1023/2046] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:50:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:50:20.931355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757f7e3-ce9d-4078-a024-fda87122dd2e None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:20.933221 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-b757f7e3-ce9d-4078-a024-fda87122dd2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:20.972011 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f872ddb-5f79-4ceb-a993-b51e521eef7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:20.974086 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f872ddb-5f79-4ceb-a993-b51e521eef7d service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:20.998755 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1024/2047] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:50:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 14:50:21.019987 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f872ddb-5f79-4ceb-a993-b51e521eef7d service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:21.035441 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757f7e3-ce9d-4078-a024-fda87122dd2e None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:21.041766 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3826275c-0bcc-49a3-a229-debd59cf5e74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:21.042672 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826275c-0bcc-49a3-a229-debd59cf5e74 service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:21.043032 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826275c-0bcc-49a3-a229-debd59cf5e74 service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:21.043409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826275c-0bcc-49a3-a229-debd59cf5e74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:21.052724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3826275c-0bcc-49a3-a229-debd59cf5e74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:21.054591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3826275c-0bcc-49a3-a229-debd59cf5e74 service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:21.065774 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1024/2048] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:50:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:50:21.099574 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f872ddb-5f79-4ceb-a993-b51e521eef7d service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:21.105020 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9dc6a-78ca-47a3-a4f2-9785ed152127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:21.105921 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9dc6a-78ca-47a3-a4f2-9785ed152127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:21.106252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9dc6a-78ca-47a3-a4f2-9785ed152127 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:21.106543 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9dc6a-78ca-47a3-a4f2-9785ed152127 admin admin] PATH_INFO: `/v3/users/ef3db5a2898a4ef2b8ca559163fcb1b8` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:21.107807 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a9dc6a-78ca-47a3-a4f2-9785ed152127 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef3db5a2898a4ef2b8ca559163fcb1b8)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:21.109966 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a9dc6a-78ca-47a3-a4f2-9785ed152127 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:21.171323 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-25a9dc6a-78ca-47a3-a4f2-9785ed152127 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef3db5a2898a4ef2b8ca559163fcb1b8'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:21.186911 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-25a9dc6a-78ca-47a3-a4f2-9785ed152127 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef3db5a2898a4ef2b8ca559163fcb1b8'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:21.188039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-25a9dc6a-78ca-47a3-a4f2-9785ed152127 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef3db5a2898a4ef2b8ca559163fcb1b8'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:21.188622 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-25a9dc6a-78ca-47a3-a4f2-9785ed152127 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef3db5a2898a4ef2b8ca559163fcb1b8'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:21.196076 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-25a9dc6a-78ca-47a3-a4f2-9785ed152127 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef3db5a2898a4ef2b8ca559163fcb1b8'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:21.220402 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1025/2049] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:50:21 2022] DELETE /identity/v3/users/ef3db5a2898a4ef2b8ca559163fcb1b8 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:50:21.227295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ffde2-eea1-4219-b3e1-5103982efe58 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:21.228107 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ffde2-eea1-4219-b3e1-5103982efe58 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:21.229099 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ffde2-eea1-4219-b3e1-5103982efe58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:21.253022 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-272ffde2-eea1-4219-b3e1-5103982efe58 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:21.254527 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-272ffde2-eea1-4219-b3e1-5103982efe58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:21.336535 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1025/2050] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:50:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:21.351082 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9dc6a-78ca-47a3-a4f2-9785ed152127 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:21.354622 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-25a9dc6a-78ca-47a3-a4f2-9785ed152127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:21.386690 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:21.469044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9dc6a-78ca-47a3-a4f2-9785ed152127 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:21.473790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d083174-eb07-4c18-8ba8-c94fa3a2fdd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:21.474636 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d083174-eb07-4c18-8ba8-c94fa3a2fdd3 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:21.475257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d083174-eb07-4c18-8ba8-c94fa3a2fdd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:21.475711 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d083174-eb07-4c18-8ba8-c94fa3a2fdd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:21.503014 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d083174-eb07-4c18-8ba8-c94fa3a2fdd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:21.505016 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d083174-eb07-4c18-8ba8-c94fa3a2fdd3 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:21.521933 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1026/2051] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:50:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:50:21.917616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ffde2-eea1-4219-b3e1-5103982efe58 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:21.919283 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-272ffde2-eea1-4219-b3e1-5103982efe58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:21.935152 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ffde2-eea1-4219-b3e1-5103982efe58 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:21.940211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5317b1-be4e-426a-a045-199bc0c2abb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:21.941189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5317b1-be4e-426a-a045-199bc0c2abb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:21.941498 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5317b1-be4e-426a-a045-199bc0c2abb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:21.941796 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5317b1-be4e-426a-a045-199bc0c2abb2 admin admin] PATH_INFO: `/v3/projects/3c44d203fb1a490f8b81af15dc9e34b5` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:21.943075 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5317b1-be4e-426a-a045-199bc0c2abb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c44d203fb1a490f8b81af15dc9e34b5)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:21.946345 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5317b1-be4e-426a-a045-199bc0c2abb2 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:22.016007 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-2c5317b1-be4e-426a-a045-199bc0c2abb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c44d203fb1a490f8b81af15dc9e34b5', 'request_id': 'req-2c5317b1-be4e-426a-a045-199bc0c2abb2'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:22.023345 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-2c5317b1-be4e-426a-a045-199bc0c2abb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c44d203fb1a490f8b81af15dc9e34b5', 'request_id': 'req-2c5317b1-be4e-426a-a045-199bc0c2abb2'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:22.035406 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-2c5317b1-be4e-426a-a045-199bc0c2abb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c44d203fb1a490f8b81af15dc9e34b5', 'request_id': 'req-2c5317b1-be4e-426a-a045-199bc0c2abb2'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:22.040405 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:22.055850 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-2c5317b1-be4e-426a-a045-199bc0c2abb2 admin admin] The token cache is being invalidate because project 3c44d203fb1a490f8b81af15dc9e34b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:50:22.057484 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-2c5317b1-be4e-426a-a045-199bc0c2abb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:22.059085 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-2c5317b1-be4e-426a-a045-199bc0c2abb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:22.063865 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1026/2052] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:50:21 2022] DELETE /identity/v3/projects/3c44d203fb1a490f8b81af15dc9e34b5 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:50:22.065748 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5fb92-e2cd-4a6c-8254-7e3acfad1427 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:22.066804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5fb92-e2cd-4a6c-8254-7e3acfad1427 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:22.067114 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5fb92-e2cd-4a6c-8254-7e3acfad1427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:22.089090 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-fce5fb92-e2cd-4a6c-8254-7e3acfad1427 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:22.090320 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-fce5fb92-e2cd-4a6c-8254-7e3acfad1427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:22.164699 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1027/2053] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:50:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:22.172476 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522bcd6a-be52-431a-90b8-79232c240311 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:22.172771 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522bcd6a-be52-431a-90b8-79232c240311 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:22.173057 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522bcd6a-be52-431a-90b8-79232c240311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:22.195100 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-522bcd6a-be52-431a-90b8-79232c240311 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:22.195788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-522bcd6a-be52-431a-90b8-79232c240311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:22.204545 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1027/2054] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:50:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:22.208832 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce5fb92-e2cd-4a6c-8254-7e3acfad1427 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:22.219488 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce5fb92-e2cd-4a6c-8254-7e3acfad1427 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:22.224981 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee90929-ef26-46f7-9a96-3be018161205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:22.225671 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee90929-ef26-46f7-9a96-3be018161205 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:22.225961 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee90929-ef26-46f7-9a96-3be018161205 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:22.226225 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee90929-ef26-46f7-9a96-3be018161205 admin admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:22.226977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee90929-ef26-46f7-9a96-3be018161205 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:50:22.227318 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee90929-ef26-46f7-9a96-3be018161205 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:22.228968 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee90929-ef26-46f7-9a96-3be018161205 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:22.244729 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1028/2055] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:50:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:50:22.248798 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522bcd6a-be52-431a-90b8-79232c240311 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:22.259802 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522bcd6a-be52-431a-90b8-79232c240311 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:22.280470 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e09bae-5614-4bd6-a854-5a5180b29149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:22.281402 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e09bae-5614-4bd6-a854-5a5180b29149 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:22.281842 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e09bae-5614-4bd6-a854-5a5180b29149 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:22.282294 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e09bae-5614-4bd6-a854-5a5180b29149 admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:22.283027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e09bae-5614-4bd6-a854-5a5180b29149 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:22.285047 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e09bae-5614-4bd6-a854-5a5180b29149 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:22.324778 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1028/2056] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:50:22 2022] POST /identity/v3/projects => generated 369 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 14:50:22.329862 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee90929-ef26-46f7-9a96-3be018161205 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:22.342080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee90929-ef26-46f7-9a96-3be018161205 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:22.406432 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031fefb9-ef72-4ae8-908b-787746620b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:22.407414 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fefb9-ef72-4ae8-908b-787746620b02 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:22.407698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fefb9-ef72-4ae8-908b-787746620b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:22.407980 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fefb9-ef72-4ae8-908b-787746620b02 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:22.408803 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031fefb9-ef72-4ae8-908b-787746620b02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:22.410833 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031fefb9-ef72-4ae8-908b-787746620b02 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:22.435082 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1029/2057] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:50:22 2022] POST /identity/v3/users => generated 285 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:50:22.436738 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e09bae-5614-4bd6-a854-5a5180b29149 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:22.448173 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e09bae-5614-4bd6-a854-5a5180b29149 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:22.453512 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1bd8d-1ede-4480-a4ab-0c794761828e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:22.454216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1bd8d-1ede-4480-a4ab-0c794761828e admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:22.454486 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1bd8d-1ede-4480-a4ab-0c794761828e admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:22.454761 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1bd8d-1ede-4480-a4ab-0c794761828e admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:22.455580 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1bd8d-1ede-4480-a4ab-0c794761828e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:22.457209 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1bd8d-1ede-4480-a4ab-0c794761828e admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:22.469306 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1029/2058] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:50:22.473657 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031fefb9-ef72-4ae8-908b-787746620b02 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:22.486919 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031fefb9-ef72-4ae8-908b-787746620b02 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:22.492838 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518173eb-c831-4907-a071-348ea0c9acc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:22.493541 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518173eb-c831-4907-a071-348ea0c9acc3 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:22.493829 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518173eb-c831-4907-a071-348ea0c9acc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:22.494110 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518173eb-c831-4907-a071-348ea0c9acc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:22.494866 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518173eb-c831-4907-a071-348ea0c9acc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:22.496555 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518173eb-c831-4907-a071-348ea0c9acc3 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:22.509224 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1030/2059] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:50:22.513547 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1bd8d-1ede-4480-a4ab-0c794761828e admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:22.525574 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1bd8d-1ede-4480-a4ab-0c794761828e admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:22.531738 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bca716-1b9a-46d1-af03-bf97a4ac5310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:22.533184 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bca716-1b9a-46d1-af03-bf97a4ac5310 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:22.533518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bca716-1b9a-46d1-af03-bf97a4ac5310 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:22.533812 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bca716-1b9a-46d1-af03-bf97a4ac5310 admin admin] PATH_INFO: `/v3/projects/ed626c0caa9941fd8e76da43dc461451/users/45c2342246b84fd4a429335e065f813d/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:22.553991 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bca716-1b9a-46d1-af03-bf97a4ac5310 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed626c0caa9941fd8e76da43dc461451, user_id=45c2342246b84fd4a429335e065f813d, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:22.556973 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bca716-1b9a-46d1-af03-bf97a4ac5310 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:22.576143 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1030/2060] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:50:22 2022] PUT /identity/v3/projects/ed626c0caa9941fd8e76da43dc461451/users/45c2342246b84fd4a429335e065f813d/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:50:22.586929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01102963-da7e-4a39-af93-81d1b9bd0ca2 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:22.587336 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01102963-da7e-4a39-af93-81d1b9bd0ca2 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:22.587609 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01102963-da7e-4a39-af93-81d1b9bd0ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:22.622268 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-01102963-da7e-4a39-af93-81d1b9bd0ca2 None None] MFA Rules not processed for user `45c2342246b84fd4a429335e065f813d`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:22.624039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-01102963-da7e-4a39-af93-81d1b9bd0ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:22.637629 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:22.706763 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1031/2061] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:50:22 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:22.719160 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bca716-1b9a-46d1-af03-bf97a4ac5310 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:22.720941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-c1bca716-1b9a-46d1-af03-bf97a4ac5310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:22.827961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bca716-1b9a-46d1-af03-bf97a4ac5310 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:22.832001 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7b765-cfab-4219-ad99-4522e9978218 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d7d59080cd4832a45dbb6c0349cf4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:22.832666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7b765-cfab-4219-ad99-4522e9978218 service glance] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:22.832957 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7b765-cfab-4219-ad99-4522e9978218 service glance] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:22.833239 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7b765-cfab-4219-ad99-4522e9978218 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:22.842959 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c7b765-cfab-4219-ad99-4522e9978218 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:22.844510 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c7b765-cfab-4219-ad99-4522e9978218 service glance] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:22.855305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1031/2062] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:50:22 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:50:30.638292 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01102963-da7e-4a39-af93-81d1b9bd0ca2 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:30.639251 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-01102963-da7e-4a39-af93-81d1b9bd0ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:30.723468 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01102963-da7e-4a39-af93-81d1b9bd0ca2 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:30.728870 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe213fdd-5fe9-489c-8535-f537327dc082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:30.729772 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe213fdd-5fe9-489c-8535-f537327dc082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:30.730056 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe213fdd-5fe9-489c-8535-f537327dc082 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:30.730333 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe213fdd-5fe9-489c-8535-f537327dc082 admin admin] PATH_INFO: `/v3/users/54f0a23979014886bfed56cf62a0fa70` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:30.731697 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe213fdd-5fe9-489c-8535-f537327dc082 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54f0a23979014886bfed56cf62a0fa70)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:30.733725 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe213fdd-5fe9-489c-8535-f537327dc082 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:30.796999 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fe213fdd-5fe9-489c-8535-f537327dc082 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54f0a23979014886bfed56cf62a0fa70'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:30.810712 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fe213fdd-5fe9-489c-8535-f537327dc082 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54f0a23979014886bfed56cf62a0fa70'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:30.811554 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fe213fdd-5fe9-489c-8535-f537327dc082 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54f0a23979014886bfed56cf62a0fa70'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:30.812276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fe213fdd-5fe9-489c-8535-f537327dc082 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54f0a23979014886bfed56cf62a0fa70'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:30.827335 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fe213fdd-5fe9-489c-8535-f537327dc082 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54f0a23979014886bfed56cf62a0fa70'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:30.844220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1032/2063] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:50:30 2022] DELETE /identity/v3/users/54f0a23979014886bfed56cf62a0fa70 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:50:30.848861 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b7004-8131-471e-b62c-38133327037c None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:30.849404 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b7004-8131-471e-b62c-38133327037c None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:30.849519 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b7004-8131-471e-b62c-38133327037c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:30.868061 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-668b7004-8131-471e-b62c-38133327037c None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:30.869264 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-668b7004-8131-471e-b62c-38133327037c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:30.937153 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1032/2064] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:50:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:30.950133 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe213fdd-5fe9-489c-8535-f537327dc082 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:30.951809 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-fe213fdd-5fe9-489c-8535-f537327dc082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:30.997376 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:31.022725 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe213fdd-5fe9-489c-8535-f537327dc082 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:31.027252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723418a-d829-4f1b-bebd-a578909dbbf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:31.028013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723418a-d829-4f1b-bebd-a578909dbbf8 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:31.028375 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723418a-d829-4f1b-bebd-a578909dbbf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:31.028970 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723418a-d829-4f1b-bebd-a578909dbbf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:31.039966 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a723418a-d829-4f1b-bebd-a578909dbbf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:31.041416 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a723418a-d829-4f1b-bebd-a578909dbbf8 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:31.051778 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1033/2065] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:50:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:50:31.440564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b7004-8131-471e-b62c-38133327037c None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:31.442028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-668b7004-8131-471e-b62c-38133327037c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:31.456057 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b7004-8131-471e-b62c-38133327037c None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:31.465083 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed904ed2-2c2a-4e42-a429-c413d53c0c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:31.466105 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed904ed2-2c2a-4e42-a429-c413d53c0c18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:31.466294 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed904ed2-2c2a-4e42-a429-c413d53c0c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:31.466577 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed904ed2-2c2a-4e42-a429-c413d53c0c18 admin admin] PATH_INFO: `/v3/projects/f0af7082b1c04ea687097e35e9050807` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:31.467839 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed904ed2-2c2a-4e42-a429-c413d53c0c18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0af7082b1c04ea687097e35e9050807)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:31.469762 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed904ed2-2c2a-4e42-a429-c413d53c0c18 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:31.489012 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:31.529565 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ed904ed2-2c2a-4e42-a429-c413d53c0c18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0af7082b1c04ea687097e35e9050807', 'request_id': 'req-ed904ed2-2c2a-4e42-a429-c413d53c0c18'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:31.537978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ed904ed2-2c2a-4e42-a429-c413d53c0c18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0af7082b1c04ea687097e35e9050807', 'request_id': 'req-ed904ed2-2c2a-4e42-a429-c413d53c0c18'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:31.548316 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ed904ed2-2c2a-4e42-a429-c413d53c0c18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0af7082b1c04ea687097e35e9050807', 'request_id': 'req-ed904ed2-2c2a-4e42-a429-c413d53c0c18'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:31.562556 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ed904ed2-2c2a-4e42-a429-c413d53c0c18 admin admin] The token cache is being invalidate because project f0af7082b1c04ea687097e35e9050807 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:50:31.562846 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ed904ed2-2c2a-4e42-a429-c413d53c0c18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:31.563504 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ed904ed2-2c2a-4e42-a429-c413d53c0c18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:31.565185 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1033/2066] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:50:31 2022] DELETE /identity/v3/projects/f0af7082b1c04ea687097e35e9050807 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:50:31.570343 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1e150-1bb1-4b69-b2e1-e37ef915c865 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:31.570636 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1e150-1bb1-4b69-b2e1-e37ef915c865 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:31.570912 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1e150-1bb1-4b69-b2e1-e37ef915c865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:31.597321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-52c1e150-1bb1-4b69-b2e1-e37ef915c865 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:31.599112 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-52c1e150-1bb1-4b69-b2e1-e37ef915c865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:31.669855 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1034/2067] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:50:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:31.678985 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf629ccf-1aa4-4eba-86ff-083a49f44529 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:31.679258 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf629ccf-1aa4-4eba-86ff-083a49f44529 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:31.679571 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf629ccf-1aa4-4eba-86ff-083a49f44529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:31.699558 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-bf629ccf-1aa4-4eba-86ff-083a49f44529 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:31.700817 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-bf629ccf-1aa4-4eba-86ff-083a49f44529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:31.709966 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1034/2068] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:50:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:31.714455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1e150-1bb1-4b69-b2e1-e37ef915c865 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:31.725946 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1e150-1bb1-4b69-b2e1-e37ef915c865 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:31.731029 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a0a8b-733b-434d-90a6-20abfb2c38aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:31.731745 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a0a8b-733b-434d-90a6-20abfb2c38aa admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:31.732037 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a0a8b-733b-434d-90a6-20abfb2c38aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:31.732316 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a0a8b-733b-434d-90a6-20abfb2c38aa admin admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:31.733061 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1a0a8b-733b-434d-90a6-20abfb2c38aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:50:31.733376 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1a0a8b-733b-434d-90a6-20abfb2c38aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:31.735013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1a0a8b-733b-434d-90a6-20abfb2c38aa admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:31.751707 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1035/2069] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:50:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:50:31.756381 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf629ccf-1aa4-4eba-86ff-083a49f44529 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:31.766089 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf629ccf-1aa4-4eba-86ff-083a49f44529 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:31.773091 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c8d7d-cb4f-4397-bd69-84a425b8430c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:31.773780 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c8d7d-cb4f-4397-bd69-84a425b8430c admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:31.774066 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c8d7d-cb4f-4397-bd69-84a425b8430c admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:31.774350 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c8d7d-cb4f-4397-bd69-84a425b8430c admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:31.775363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c8d7d-cb4f-4397-bd69-84a425b8430c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:31.778224 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c8d7d-cb4f-4397-bd69-84a425b8430c admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:31.819741 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1035/2070] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:50:31 2022] POST /identity/v3/projects => generated 387 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:50:31.825504 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a0a8b-733b-434d-90a6-20abfb2c38aa admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:31.837880 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a0a8b-733b-434d-90a6-20abfb2c38aa admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:31.898090 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de9e9d-475a-4cc5-9479-521e54566fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:31.898811 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de9e9d-475a-4cc5-9479-521e54566fb7 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:31.899107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de9e9d-475a-4cc5-9479-521e54566fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:31.899394 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de9e9d-475a-4cc5-9479-521e54566fb7 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:31.900093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de9e9d-475a-4cc5-9479-521e54566fb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:31.902060 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de9e9d-475a-4cc5-9479-521e54566fb7 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:31.926584 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1036/2071] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:50:31 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:50:31.931093 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c8d7d-cb4f-4397-bd69-84a425b8430c admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:31.941977 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c8d7d-cb4f-4397-bd69-84a425b8430c admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:31.947163 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c149d9-95c5-4bf9-a524-d9b6d6ec61c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:31.948161 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c149d9-95c5-4bf9-a524-d9b6d6ec61c5 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:31.948161 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c149d9-95c5-4bf9-a524-d9b6d6ec61c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:31.948694 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c149d9-95c5-4bf9-a524-d9b6d6ec61c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:31.949115 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c149d9-95c5-4bf9-a524-d9b6d6ec61c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:31.950729 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c149d9-95c5-4bf9-a524-d9b6d6ec61c5 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:31.962185 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1036/2072] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:50:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:50:31.966710 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de9e9d-475a-4cc5-9479-521e54566fb7 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:31.977452 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de9e9d-475a-4cc5-9479-521e54566fb7 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:31.983383 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb695e61-9766-4fc4-b3da-7f4c90bcdce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:31.984116 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb695e61-9766-4fc4-b3da-7f4c90bcdce5 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:31.984434 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb695e61-9766-4fc4-b3da-7f4c90bcdce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:31.984713 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb695e61-9766-4fc4-b3da-7f4c90bcdce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:31.985472 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb695e61-9766-4fc4-b3da-7f4c90bcdce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:31.987262 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb695e61-9766-4fc4-b3da-7f4c90bcdce5 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:31.998972 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1037/2073] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:50:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:50:32.003682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c149d9-95c5-4bf9-a524-d9b6d6ec61c5 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:32.014750 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c149d9-95c5-4bf9-a524-d9b6d6ec61c5 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:32.020788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd796054-85c4-4b0d-a6a0-07d3ed2a2fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:32.021707 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd796054-85c4-4b0d-a6a0-07d3ed2a2fa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:32.021991 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd796054-85c4-4b0d-a6a0-07d3ed2a2fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:32.022274 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd796054-85c4-4b0d-a6a0-07d3ed2a2fa0 admin admin] PATH_INFO: `/v3/projects/eab4d8aaff4a45eab383a1cfe4bf87c6/users/4e95787aa2544c91815abf5a357ed923/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:32.040038 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd796054-85c4-4b0d-a6a0-07d3ed2a2fa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eab4d8aaff4a45eab383a1cfe4bf87c6, user_id=4e95787aa2544c91815abf5a357ed923, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:32.042842 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd796054-85c4-4b0d-a6a0-07d3ed2a2fa0 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:32.060323 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1037/2074] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:50:32 2022] PUT /identity/v3/projects/eab4d8aaff4a45eab383a1cfe4bf87c6/users/4e95787aa2544c91815abf5a357ed923/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:50:32.071464 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc6661-e620-463d-a1b2-8724695345fe None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:32.071705 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc6661-e620-463d-a1b2-8724695345fe None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:32.071933 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc6661-e620-463d-a1b2-8724695345fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:32.104828 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-a3cc6661-e620-463d-a1b2-8724695345fe None None] MFA Rules not processed for user `4e95787aa2544c91815abf5a357ed923`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:32.106041 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-a3cc6661-e620-463d-a1b2-8724695345fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:32.153190 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:32.191824 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1038/2075] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:50:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:32.197018 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd796054-85c4-4b0d-a6a0-07d3ed2a2fa0 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:32.283126 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd796054-85c4-4b0d-a6a0-07d3ed2a2fa0 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:32.289313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74505148-9626-4564-82d3-55058c34f5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:32.290038 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74505148-9626-4564-82d3-55058c34f5da admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:32.290323 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74505148-9626-4564-82d3-55058c34f5da admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:32.290597 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74505148-9626-4564-82d3-55058c34f5da admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:32.291437 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74505148-9626-4564-82d3-55058c34f5da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:32.293375 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74505148-9626-4564-82d3-55058c34f5da admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:32.326781 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1038/2076] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:50:32 2022] POST /identity/v3/projects => generated 387 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 14:50:32.332392 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc6661-e620-463d-a1b2-8724695345fe None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:32.344584 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc6661-e620-463d-a1b2-8724695345fe None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:32.412757 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b260101-f156-4fcf-ba21-1c36f5a7e4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:32.413471 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b260101-f156-4fcf-ba21-1c36f5a7e4e2 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:32.413779 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b260101-f156-4fcf-ba21-1c36f5a7e4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:32.414048 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b260101-f156-4fcf-ba21-1c36f5a7e4e2 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:32.414749 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b260101-f156-4fcf-ba21-1c36f5a7e4e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:32.416719 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b260101-f156-4fcf-ba21-1c36f5a7e4e2 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:32.440949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1039/2077] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:50:32 2022] POST /identity/v3/users => generated 300 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:50:32.447013 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74505148-9626-4564-82d3-55058c34f5da admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:32.464348 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74505148-9626-4564-82d3-55058c34f5da admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:32.472766 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5f075-5123-40e2-bc11-0e8884029a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:32.473944 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5f075-5123-40e2-bc11-0e8884029a6b admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:32.474474 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5f075-5123-40e2-bc11-0e8884029a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:32.474980 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5f075-5123-40e2-bc11-0e8884029a6b admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:32.476394 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed5f075-5123-40e2-bc11-0e8884029a6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:32.479280 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed5f075-5123-40e2-bc11-0e8884029a6b admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:32.495875 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1039/2078] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:50:32.500164 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b260101-f156-4fcf-ba21-1c36f5a7e4e2 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:32.510542 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b260101-f156-4fcf-ba21-1c36f5a7e4e2 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:32.515423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26013ff-26c8-46e0-8f18-7d16908308c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:32.516361 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26013ff-26c8-46e0-8f18-7d16908308c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:32.516925 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26013ff-26c8-46e0-8f18-7d16908308c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:32.517203 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26013ff-26c8-46e0-8f18-7d16908308c9 admin admin] PATH_INFO: `/v3/projects/806d8aa3a9314688ba3a8bc9643e65a9/users/19ac60604dd04d15a64b0ab5b7de6749/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:32.534836 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26013ff-26c8-46e0-8f18-7d16908308c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=806d8aa3a9314688ba3a8bc9643e65a9, user_id=19ac60604dd04d15a64b0ab5b7de6749, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:32.537581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26013ff-26c8-46e0-8f18-7d16908308c9 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:32.552883 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1040/2079] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:50:32 2022] PUT /identity/v3/projects/806d8aa3a9314688ba3a8bc9643e65a9/users/19ac60604dd04d15a64b0ab5b7de6749/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 14:50:32.557008 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5f075-5123-40e2-bc11-0e8884029a6b admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:32.567996 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5f075-5123-40e2-bc11-0e8884029a6b admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:32.573235 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:32.643902 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba42eca-95a2-40ee-9549-9c5dba7081ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:32.644586 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba42eca-95a2-40ee-9549-9c5dba7081ef admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:32.644870 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba42eca-95a2-40ee-9549-9c5dba7081ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:32.645148 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba42eca-95a2-40ee-9549-9c5dba7081ef admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:32.645919 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba42eca-95a2-40ee-9549-9c5dba7081ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:32.647565 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba42eca-95a2-40ee-9549-9c5dba7081ef admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:32.658630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1040/2080] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:50:32.663657 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26013ff-26c8-46e0-8f18-7d16908308c9 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:32.674595 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26013ff-26c8-46e0-8f18-7d16908308c9 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:32.679909 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a884679-9738-41e0-8d46-6565795908c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:32.680800 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a884679-9738-41e0-8d46-6565795908c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:32.681104 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a884679-9738-41e0-8d46-6565795908c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:32.681380 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a884679-9738-41e0-8d46-6565795908c2 admin admin] PATH_INFO: `/v3/projects/806d8aa3a9314688ba3a8bc9643e65a9/users/19ac60604dd04d15a64b0ab5b7de6749/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:32.683618 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a884679-9738-41e0-8d46-6565795908c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=806d8aa3a9314688ba3a8bc9643e65a9, user_id=19ac60604dd04d15a64b0ab5b7de6749, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:32.686518 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a884679-9738-41e0-8d46-6565795908c2 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:32.699770 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1041/2081] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:50:32 2022] PUT /identity/v3/projects/806d8aa3a9314688ba3a8bc9643e65a9/users/19ac60604dd04d15a64b0ab5b7de6749/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:50:32.710497 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b313286-84da-4789-a773-932217dda631 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:32.710788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b313286-84da-4789-a773-932217dda631 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:32.711074 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b313286-84da-4789-a773-932217dda631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:32.745473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-5b313286-84da-4789-a773-932217dda631 None None] MFA Rules not processed for user `19ac60604dd04d15a64b0ab5b7de6749`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:32.746739 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-5b313286-84da-4789-a773-932217dda631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:32.828375 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1041/2082] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:50:32 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:32.836523 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a884679-9738-41e0-8d46-6565795908c2 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:32.838390 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-0a884679-9738-41e0-8d46-6565795908c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:32.916027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a884679-9738-41e0-8d46-6565795908c2 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:32.921064 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0296cc8f-9570-4cdf-84a6-731abf94e467 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:32.921850 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296cc8f-9570-4cdf-84a6-731abf94e467 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:32.922206 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296cc8f-9570-4cdf-84a6-731abf94e467 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:32.922544 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296cc8f-9570-4cdf-84a6-731abf94e467 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:32.937779 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0296cc8f-9570-4cdf-84a6-731abf94e467 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:32.939613 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0296cc8f-9570-4cdf-84a6-731abf94e467 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:32.952054 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1042/2083] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:50:32 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 14:50:32.985201 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b313286-84da-4789-a773-932217dda631 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:33.070273 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b313286-84da-4789-a773-932217dda631 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:33.076252 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02cabe-b279-44db-8e58-c2d698e5d17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:33.077166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02cabe-b279-44db-8e58-c2d698e5d17b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:33.077450 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02cabe-b279-44db-8e58-c2d698e5d17b admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:33.077730 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02cabe-b279-44db-8e58-c2d698e5d17b admin admin] PATH_INFO: `/v3/users/4e95787aa2544c91815abf5a357ed923` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:33.079107 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d02cabe-b279-44db-8e58-c2d698e5d17b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e95787aa2544c91815abf5a357ed923)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:33.081068 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d02cabe-b279-44db-8e58-c2d698e5d17b admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:33.124955 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:33.141151 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-7d02cabe-b279-44db-8e58-c2d698e5d17b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e95787aa2544c91815abf5a357ed923'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:33.153275 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-7d02cabe-b279-44db-8e58-c2d698e5d17b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e95787aa2544c91815abf5a357ed923'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:33.154375 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-7d02cabe-b279-44db-8e58-c2d698e5d17b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e95787aa2544c91815abf5a357ed923'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:33.155089 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-7d02cabe-b279-44db-8e58-c2d698e5d17b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e95787aa2544c91815abf5a357ed923'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:33.169914 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-7d02cabe-b279-44db-8e58-c2d698e5d17b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e95787aa2544c91815abf5a357ed923'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:33.184010 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1042/2084] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:50:32 2022] DELETE /identity/v3/users/4e95787aa2544c91815abf5a357ed923 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:50:33.188662 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c9a7f-f506-49da-a2b1-8ed1da6d7891 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:33.189135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c9a7f-f506-49da-a2b1-8ed1da6d7891 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:33.189555 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c9a7f-f506-49da-a2b1-8ed1da6d7891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:33.216299 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-753c9a7f-f506-49da-a2b1-8ed1da6d7891 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:33.217514 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-753c9a7f-f506-49da-a2b1-8ed1da6d7891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:33.238634 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:33.288858 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1043/2085] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:50:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:50:33.301710 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02cabe-b279-44db-8e58-c2d698e5d17b admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:33.304688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-7d02cabe-b279-44db-8e58-c2d698e5d17b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:33.399124 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02cabe-b279-44db-8e58-c2d698e5d17b admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:33.403034 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c71a612-5cd8-4b3d-9cea-4312cfe6de9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:33.404152 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71a612-5cd8-4b3d-9cea-4312cfe6de9a service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:33.404538 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71a612-5cd8-4b3d-9cea-4312cfe6de9a service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:33.404937 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71a612-5cd8-4b3d-9cea-4312cfe6de9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:33.415758 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c71a612-5cd8-4b3d-9cea-4312cfe6de9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:33.417375 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c71a612-5cd8-4b3d-9cea-4312cfe6de9a service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:33.429455 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1043/2086] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:50:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:50:33.761265 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c9a7f-f506-49da-a2b1-8ed1da6d7891 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:33.762783 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-753c9a7f-f506-49da-a2b1-8ed1da6d7891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:33.775461 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c9a7f-f506-49da-a2b1-8ed1da6d7891 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:33.781109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a073de-925c-4404-9f8d-a50ab1a57617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:33.782047 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a073de-925c-4404-9f8d-a50ab1a57617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:33.782391 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a073de-925c-4404-9f8d-a50ab1a57617 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:33.782732 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a073de-925c-4404-9f8d-a50ab1a57617 admin admin] PATH_INFO: `/v3/projects/eab4d8aaff4a45eab383a1cfe4bf87c6` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:33.784077 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a073de-925c-4404-9f8d-a50ab1a57617 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eab4d8aaff4a45eab383a1cfe4bf87c6)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:33.786760 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a073de-925c-4404-9f8d-a50ab1a57617 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:33.844811 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-c4a073de-925c-4404-9f8d-a50ab1a57617 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eab4d8aaff4a45eab383a1cfe4bf87c6', 'request_id': 'req-c4a073de-925c-4404-9f8d-a50ab1a57617'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:33.850760 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-c4a073de-925c-4404-9f8d-a50ab1a57617 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eab4d8aaff4a45eab383a1cfe4bf87c6', 'request_id': 'req-c4a073de-925c-4404-9f8d-a50ab1a57617'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:33.861118 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-c4a073de-925c-4404-9f8d-a50ab1a57617 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eab4d8aaff4a45eab383a1cfe4bf87c6', 'request_id': 'req-c4a073de-925c-4404-9f8d-a50ab1a57617'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:33.872985 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-c4a073de-925c-4404-9f8d-a50ab1a57617 admin admin] The token cache is being invalidate because project eab4d8aaff4a45eab383a1cfe4bf87c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:50:33.873282 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-c4a073de-925c-4404-9f8d-a50ab1a57617 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:33.873928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-c4a073de-925c-4404-9f8d-a50ab1a57617 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:33.875534 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1044/2087] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:50:33 2022] DELETE /identity/v3/projects/eab4d8aaff4a45eab383a1cfe4bf87c6 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:50:33.879816 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c71a612-5cd8-4b3d-9cea-4312cfe6de9a service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:33.889669 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c71a612-5cd8-4b3d-9cea-4312cfe6de9a service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:33.964373 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf74e24-5b54-4371-a9cc-bcaf98df3adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:33.965268 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf74e24-5b54-4371-a9cc-bcaf98df3adb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:33.965557 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf74e24-5b54-4371-a9cc-bcaf98df3adb admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:33.965845 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf74e24-5b54-4371-a9cc-bcaf98df3adb admin admin] PATH_INFO: `/v3/users/19ac60604dd04d15a64b0ab5b7de6749` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:33.967101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf74e24-5b54-4371-a9cc-bcaf98df3adb admin admin] RBAC: Authorizing `identity:delete_user(user_id=19ac60604dd04d15a64b0ab5b7de6749)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:33.969047 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf74e24-5b54-4371-a9cc-bcaf98df3adb admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:34.021517 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-4bf74e24-5b54-4371-a9cc-bcaf98df3adb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19ac60604dd04d15a64b0ab5b7de6749'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:34.027623 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:34.036662 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-4bf74e24-5b54-4371-a9cc-bcaf98df3adb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19ac60604dd04d15a64b0ab5b7de6749'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:34.037516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-4bf74e24-5b54-4371-a9cc-bcaf98df3adb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19ac60604dd04d15a64b0ab5b7de6749'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:34.038805 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-4bf74e24-5b54-4371-a9cc-bcaf98df3adb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19ac60604dd04d15a64b0ab5b7de6749'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:34.046673 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-4bf74e24-5b54-4371-a9cc-bcaf98df3adb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19ac60604dd04d15a64b0ab5b7de6749'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:34.061126 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1044/2088] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:50:33 2022] DELETE /identity/v3/users/19ac60604dd04d15a64b0ab5b7de6749 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:50:34.349207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a073de-925c-4404-9f8d-a50ab1a57617 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:34.350810 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-c4a073de-925c-4404-9f8d-a50ab1a57617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:34.365569 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a073de-925c-4404-9f8d-a50ab1a57617 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:34.426580 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb94ec8-f9f7-4e99-84cc-15ffc3e3a013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:34.427475 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb94ec8-f9f7-4e99-84cc-15ffc3e3a013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:34.427758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb94ec8-f9f7-4e99-84cc-15ffc3e3a013 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:34.428033 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb94ec8-f9f7-4e99-84cc-15ffc3e3a013 admin admin] PATH_INFO: `/v3/projects/806d8aa3a9314688ba3a8bc9643e65a9` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:34.429258 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb94ec8-f9f7-4e99-84cc-15ffc3e3a013 admin admin] RBAC: Authorizing `identity:delete_project(project_id=806d8aa3a9314688ba3a8bc9643e65a9)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:34.431228 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb94ec8-f9f7-4e99-84cc-15ffc3e3a013 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:34.485520 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-4cb94ec8-f9f7-4e99-84cc-15ffc3e3a013 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '806d8aa3a9314688ba3a8bc9643e65a9', 'request_id': 'req-4cb94ec8-f9f7-4e99-84cc-15ffc3e3a013'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:34.487822 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:34.493162 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-4cb94ec8-f9f7-4e99-84cc-15ffc3e3a013 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '806d8aa3a9314688ba3a8bc9643e65a9', 'request_id': 'req-4cb94ec8-f9f7-4e99-84cc-15ffc3e3a013'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:34.504058 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-4cb94ec8-f9f7-4e99-84cc-15ffc3e3a013 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '806d8aa3a9314688ba3a8bc9643e65a9', 'request_id': 'req-4cb94ec8-f9f7-4e99-84cc-15ffc3e3a013'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:34.517823 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-4cb94ec8-f9f7-4e99-84cc-15ffc3e3a013 admin admin] The token cache is being invalidate because project 806d8aa3a9314688ba3a8bc9643e65a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:50:34.518110 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-4cb94ec8-f9f7-4e99-84cc-15ffc3e3a013 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:34.518784 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-4cb94ec8-f9f7-4e99-84cc-15ffc3e3a013 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:34.520465 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1045/2089] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:50:34 2022] DELETE /identity/v3/projects/806d8aa3a9314688ba3a8bc9643e65a9 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:50:34.525699 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a7d14-755d-4d42-bb2f-5e75cf9f108f None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:34.526051 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a7d14-755d-4d42-bb2f-5e75cf9f108f None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:34.526420 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a7d14-755d-4d42-bb2f-5e75cf9f108f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:34.547912 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-079a7d14-755d-4d42-bb2f-5e75cf9f108f None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:34.549239 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-079a7d14-755d-4d42-bb2f-5e75cf9f108f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:34.611779 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1045/2090] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:50:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:34.620832 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f48d14-675a-4e5a-ab07-ee48121f1bd7 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:34.621196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f48d14-675a-4e5a-ab07-ee48121f1bd7 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:34.621485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f48d14-675a-4e5a-ab07-ee48121f1bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:34.642085 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-80f48d14-675a-4e5a-ab07-ee48121f1bd7 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:34.643557 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-80f48d14-675a-4e5a-ab07-ee48121f1bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:34.656324 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1046/2091] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:50:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:34.657552 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079a7d14-755d-4d42-bb2f-5e75cf9f108f None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:34.667929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079a7d14-755d-4d42-bb2f-5e75cf9f108f None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:34.672875 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc06675-1549-4724-9726-1d7fa901358c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:34.673579 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc06675-1549-4724-9726-1d7fa901358c admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:34.673889 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc06675-1549-4724-9726-1d7fa901358c admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:34.674185 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc06675-1549-4724-9726-1d7fa901358c admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:34.679278 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc06675-1549-4724-9726-1d7fa901358c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:50:34.679278 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc06675-1549-4724-9726-1d7fa901358c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:34.679278 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc06675-1549-4724-9726-1d7fa901358c admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:34.690623 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1046/2092] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:50:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:50:34.695028 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f48d14-675a-4e5a-ab07-ee48121f1bd7 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:34.704401 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f48d14-675a-4e5a-ab07-ee48121f1bd7 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:34.709291 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20728efd-021c-4708-91da-25480660224b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:34.709954 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20728efd-021c-4708-91da-25480660224b admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:34.710240 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20728efd-021c-4708-91da-25480660224b admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:34.710514 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20728efd-021c-4708-91da-25480660224b admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:34.711259 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20728efd-021c-4708-91da-25480660224b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:34.713171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20728efd-021c-4708-91da-25480660224b admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:34.742267 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1047/2093] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:50:34 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:50:34.746391 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc06675-1549-4724-9726-1d7fa901358c admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:34.756618 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc06675-1549-4724-9726-1d7fa901358c admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:34.818048 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ce5f1-5db5-4906-9b1e-d6b749153d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:34.818731 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ce5f1-5db5-4906-9b1e-d6b749153d77 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:34.819023 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ce5f1-5db5-4906-9b1e-d6b749153d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:34.819340 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ce5f1-5db5-4906-9b1e-d6b749153d77 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:34.820037 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30ce5f1-5db5-4906-9b1e-d6b749153d77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:34.821999 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30ce5f1-5db5-4906-9b1e-d6b749153d77 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:34.844483 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1047/2094] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:50:34 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:50:34.849000 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20728efd-021c-4708-91da-25480660224b admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:34.862086 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20728efd-021c-4708-91da-25480660224b admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:34.868855 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13702ab-3869-49ea-91d8-fe7bba09914b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:34.869552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13702ab-3869-49ea-91d8-fe7bba09914b admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:34.869870 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13702ab-3869-49ea-91d8-fe7bba09914b admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:34.870261 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13702ab-3869-49ea-91d8-fe7bba09914b admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:34.871013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13702ab-3869-49ea-91d8-fe7bba09914b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:34.873090 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13702ab-3869-49ea-91d8-fe7bba09914b admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:34.886937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1048/2095] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:50:34.891333 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ce5f1-5db5-4906-9b1e-d6b749153d77 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:34.902520 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ce5f1-5db5-4906-9b1e-d6b749153d77 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:34.908389 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f4aa66-6ac0-47d8-b41e-942edc594615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:34.909331 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f4aa66-6ac0-47d8-b41e-942edc594615 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:34.909331 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f4aa66-6ac0-47d8-b41e-942edc594615 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:34.909634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f4aa66-6ac0-47d8-b41e-942edc594615 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:34.910355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f4aa66-6ac0-47d8-b41e-942edc594615 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:34.911957 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f4aa66-6ac0-47d8-b41e-942edc594615 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:34.924921 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1048/2096] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:50:34.930152 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13702ab-3869-49ea-91d8-fe7bba09914b admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:34.943158 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13702ab-3869-49ea-91d8-fe7bba09914b admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:34.950327 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee342d8a-bf6d-4855-bff2-f50f9e3dc188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:34.950327 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee342d8a-bf6d-4855-bff2-f50f9e3dc188 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:34.950327 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee342d8a-bf6d-4855-bff2-f50f9e3dc188 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:34.950327 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee342d8a-bf6d-4855-bff2-f50f9e3dc188 admin admin] PATH_INFO: `/v3/projects/be6cb72c4d5b4b63afea584e8e5bb933/users/68a3f28d83a440c5a8b91679ab1c61c8/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:34.966328 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee342d8a-bf6d-4855-bff2-f50f9e3dc188 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be6cb72c4d5b4b63afea584e8e5bb933, user_id=68a3f28d83a440c5a8b91679ab1c61c8, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:34.969093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee342d8a-bf6d-4855-bff2-f50f9e3dc188 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:34.983876 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1049/2097] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:50:34 2022] PUT /identity/v3/projects/be6cb72c4d5b4b63afea584e8e5bb933/users/68a3f28d83a440c5a8b91679ab1c61c8/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:50:34.994677 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803c9cd-929c-4867-891b-ab78ad449506 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:34.994916 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803c9cd-929c-4867-891b-ab78ad449506 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:34.995227 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803c9cd-929c-4867-891b-ab78ad449506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:35.021981 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-4803c9cd-929c-4867-891b-ab78ad449506 None None] MFA Rules not processed for user `68a3f28d83a440c5a8b91679ab1c61c8`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:35.023329 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-4803c9cd-929c-4867-891b-ab78ad449506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:35.051302 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:35.091761 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1049/2098] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:50:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:35.096415 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee342d8a-bf6d-4855-bff2-f50f9e3dc188 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:35.163037 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee342d8a-bf6d-4855-bff2-f50f9e3dc188 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:35.167977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1c8b5-a1c0-4283-9606-e32d140ab70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:35.168620 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1c8b5-a1c0-4283-9606-e32d140ab70f admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:35.168925 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1c8b5-a1c0-4283-9606-e32d140ab70f admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:35.169186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1c8b5-a1c0-4283-9606-e32d140ab70f admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:35.169915 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa1c8b5-a1c0-4283-9606-e32d140ab70f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:35.171860 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa1c8b5-a1c0-4283-9606-e32d140ab70f admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:35.199130 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1050/2099] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:50:35 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 14:50:35.203157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803c9cd-929c-4867-891b-ab78ad449506 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:35.225601 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803c9cd-929c-4867-891b-ab78ad449506 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:35.313010 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5175370-29db-47e9-9dcd-286fcdee1c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:35.313706 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5175370-29db-47e9-9dcd-286fcdee1c7e admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:35.313992 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5175370-29db-47e9-9dcd-286fcdee1c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:35.314299 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5175370-29db-47e9-9dcd-286fcdee1c7e admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:35.315094 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5175370-29db-47e9-9dcd-286fcdee1c7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:35.317085 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5175370-29db-47e9-9dcd-286fcdee1c7e admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:35.343206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1050/2100] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:50:35 2022] POST /identity/v3/users => generated 298 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:50:35.345313 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1c8b5-a1c0-4283-9606-e32d140ab70f admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:35.360017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1c8b5-a1c0-4283-9606-e32d140ab70f admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:35.366882 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9246eb1-7bc0-4221-b9a8-c248621409b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:35.367607 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9246eb1-7bc0-4221-b9a8-c248621409b0 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:35.367889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9246eb1-7bc0-4221-b9a8-c248621409b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:35.368203 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9246eb1-7bc0-4221-b9a8-c248621409b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:35.368952 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9246eb1-7bc0-4221-b9a8-c248621409b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:35.370609 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9246eb1-7bc0-4221-b9a8-c248621409b0 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:35.383418 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1051/2101] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:50:35.387927 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5175370-29db-47e9-9dcd-286fcdee1c7e admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:35.398667 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5175370-29db-47e9-9dcd-286fcdee1c7e admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:35.404183 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23334442-92a5-4fc1-b9ef-0b3956555f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:35.405086 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23334442-92a5-4fc1-b9ef-0b3956555f8f admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:35.405369 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23334442-92a5-4fc1-b9ef-0b3956555f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:35.405648 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23334442-92a5-4fc1-b9ef-0b3956555f8f admin admin] PATH_INFO: `/v3/projects/f3080730558f4971acb5eeab62199441/users/25c561ce2ff44765b85dda08b73a6cc8/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:35.423684 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23334442-92a5-4fc1-b9ef-0b3956555f8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3080730558f4971acb5eeab62199441, user_id=25c561ce2ff44765b85dda08b73a6cc8, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:35.426477 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23334442-92a5-4fc1-b9ef-0b3956555f8f admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:35.449295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1051/2102] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:50:35 2022] PUT /identity/v3/projects/f3080730558f4971acb5eeab62199441/users/25c561ce2ff44765b85dda08b73a6cc8/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:50:35.454858 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9246eb1-7bc0-4221-b9a8-c248621409b0 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:35.465518 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9246eb1-7bc0-4221-b9a8-c248621409b0 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:35.521323 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:35.534201 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786c569-eb2e-4ffb-afc0-d27b56338998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:35.535019 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786c569-eb2e-4ffb-afc0-d27b56338998 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:35.535378 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786c569-eb2e-4ffb-afc0-d27b56338998 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:35.536181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786c569-eb2e-4ffb-afc0-d27b56338998 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:35.536968 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4786c569-eb2e-4ffb-afc0-d27b56338998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:35.538678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4786c569-eb2e-4ffb-afc0-d27b56338998 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:35.551269 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1052/2103] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:50:35.555432 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23334442-92a5-4fc1-b9ef-0b3956555f8f admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:35.572841 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23334442-92a5-4fc1-b9ef-0b3956555f8f admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:35.578693 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8b3ce-aadb-43bb-abf4-b38bcdca280d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:35.580249 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8b3ce-aadb-43bb-abf4-b38bcdca280d admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:35.580531 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8b3ce-aadb-43bb-abf4-b38bcdca280d admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:35.580810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8b3ce-aadb-43bb-abf4-b38bcdca280d admin admin] PATH_INFO: `/v3/projects/f3080730558f4971acb5eeab62199441/users/25c561ce2ff44765b85dda08b73a6cc8/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:35.582958 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8b3ce-aadb-43bb-abf4-b38bcdca280d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3080730558f4971acb5eeab62199441, user_id=25c561ce2ff44765b85dda08b73a6cc8, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:35.585638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8b3ce-aadb-43bb-abf4-b38bcdca280d admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:35.599350 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1052/2104] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:50:35 2022] PUT /identity/v3/projects/f3080730558f4971acb5eeab62199441/users/25c561ce2ff44765b85dda08b73a6cc8/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:50:35.611573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d60e2-38d5-4ae3-a95e-8e78c36de18e None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:35.611844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d60e2-38d5-4ae3-a95e-8e78c36de18e None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:35.612176 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d60e2-38d5-4ae3-a95e-8e78c36de18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:35.644520 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-8b0d60e2-38d5-4ae3-a95e-8e78c36de18e None None] MFA Rules not processed for user `25c561ce2ff44765b85dda08b73a6cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:35.645795 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-8b0d60e2-38d5-4ae3-a95e-8e78c36de18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:35.722277 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1053/2105] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:50:35 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:35.734734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8b3ce-aadb-43bb-abf4-b38bcdca280d admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:35.736595 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-62f8b3ce-aadb-43bb-abf4-b38bcdca280d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:35.799750 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8b3ce-aadb-43bb-abf4-b38bcdca280d admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:35.805050 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7851b15a-6363-4be9-8ee9-2065593a50c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:35.805892 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7851b15a-6363-4be9-8ee9-2065593a50c7 service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:35.806363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7851b15a-6363-4be9-8ee9-2065593a50c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:35.806899 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7851b15a-6363-4be9-8ee9-2065593a50c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:35.817822 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7851b15a-6363-4be9-8ee9-2065593a50c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:35.819322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7851b15a-6363-4be9-8ee9-2065593a50c7 service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:35.830146 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1053/2106] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:50:35 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:50:35.918087 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d60e2-38d5-4ae3-a95e-8e78c36de18e None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:35.928794 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d60e2-38d5-4ae3-a95e-8e78c36de18e None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:35.932670 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e9cbc-a3a3-4aff-819c-a038a790a426 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:35.933323 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e9cbc-a3a3-4aff-819c-a038a790a426 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:35.933734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e9cbc-a3a3-4aff-819c-a038a790a426 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:35.933980 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e9cbc-a3a3-4aff-819c-a038a790a426 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:35.999195 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e9cbc-a3a3-4aff-819c-a038a790a426 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:36.000873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e9cbc-a3a3-4aff-819c-a038a790a426 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:36.013800 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1054/2107] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:50:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:50:36.302658 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7851b15a-6363-4be9-8ee9-2065593a50c7 service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:36.332117 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:36.376391 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7851b15a-6363-4be9-8ee9-2065593a50c7 service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:36.381214 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a84aaa-f568-49fc-b2a5-a46c0ac3e29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:36.382143 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a84aaa-f568-49fc-b2a5-a46c0ac3e29e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:36.382507 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a84aaa-f568-49fc-b2a5-a46c0ac3e29e admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:36.382824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a84aaa-f568-49fc-b2a5-a46c0ac3e29e admin admin] PATH_INFO: `/v3/users/68a3f28d83a440c5a8b91679ab1c61c8` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:36.384137 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a84aaa-f568-49fc-b2a5-a46c0ac3e29e admin admin] RBAC: Authorizing `identity:delete_user(user_id=68a3f28d83a440c5a8b91679ab1c61c8)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:36.386123 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a84aaa-f568-49fc-b2a5-a46c0ac3e29e admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:36.440992 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-f9a84aaa-f568-49fc-b2a5-a46c0ac3e29e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68a3f28d83a440c5a8b91679ab1c61c8'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:36.454431 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-f9a84aaa-f568-49fc-b2a5-a46c0ac3e29e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68a3f28d83a440c5a8b91679ab1c61c8'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:36.455255 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-f9a84aaa-f568-49fc-b2a5-a46c0ac3e29e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68a3f28d83a440c5a8b91679ab1c61c8'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:36.455860 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-f9a84aaa-f568-49fc-b2a5-a46c0ac3e29e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68a3f28d83a440c5a8b91679ab1c61c8'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:36.464101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-f9a84aaa-f568-49fc-b2a5-a46c0ac3e29e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68a3f28d83a440c5a8b91679ab1c61c8'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:36.484787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1054/2108] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:50:36 2022] DELETE /identity/v3/users/68a3f28d83a440c5a8b91679ab1c61c8 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:50:36.487116 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06427a-d378-4f09-8e8a-d83c6c71e882 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:36.487419 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06427a-d378-4f09-8e8a-d83c6c71e882 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:36.487752 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06427a-d378-4f09-8e8a-d83c6c71e882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:36.508835 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-9e06427a-d378-4f09-8e8a-d83c6c71e882 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:36.510203 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-9e06427a-d378-4f09-8e8a-d83c6c71e882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:36.588962 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1055/2109] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:50:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:50:36.600831 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a84aaa-f568-49fc-b2a5-a46c0ac3e29e admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:36.602464 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-f9a84aaa-f568-49fc-b2a5-a46c0ac3e29e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:36.678461 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a84aaa-f568-49fc-b2a5-a46c0ac3e29e admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:36.683089 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141fc0d-9576-4823-a211-abc78731472f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:36.683775 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141fc0d-9576-4823-a211-abc78731472f service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:36.684066 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141fc0d-9576-4823-a211-abc78731472f service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:36.684344 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141fc0d-9576-4823-a211-abc78731472f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:36.694942 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b141fc0d-9576-4823-a211-abc78731472f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:36.697145 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b141fc0d-9576-4823-a211-abc78731472f service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:36.708446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1055/2110] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:50:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:50:37.081867 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e06427a-d378-4f09-8e8a-d83c6c71e882 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:37.083826 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-9e06427a-d378-4f09-8e8a-d83c6c71e882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:37.107582 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e06427a-d378-4f09-8e8a-d83c6c71e882 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:37.113560 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57091d8e-959c-4b98-9dac-164cd28af4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:37.114389 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57091d8e-959c-4b98-9dac-164cd28af4a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:37.114680 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57091d8e-959c-4b98-9dac-164cd28af4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:37.114962 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57091d8e-959c-4b98-9dac-164cd28af4a5 admin admin] PATH_INFO: `/v3/projects/be6cb72c4d5b4b63afea584e8e5bb933` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:37.116279 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57091d8e-959c-4b98-9dac-164cd28af4a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be6cb72c4d5b4b63afea584e8e5bb933)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:37.118609 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57091d8e-959c-4b98-9dac-164cd28af4a5 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:37.125951 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:37.205984 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-57091d8e-959c-4b98-9dac-164cd28af4a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be6cb72c4d5b4b63afea584e8e5bb933', 'request_id': 'req-57091d8e-959c-4b98-9dac-164cd28af4a5'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:37.217321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-57091d8e-959c-4b98-9dac-164cd28af4a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be6cb72c4d5b4b63afea584e8e5bb933', 'request_id': 'req-57091d8e-959c-4b98-9dac-164cd28af4a5'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:37.226209 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-57091d8e-959c-4b98-9dac-164cd28af4a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be6cb72c4d5b4b63afea584e8e5bb933', 'request_id': 'req-57091d8e-959c-4b98-9dac-164cd28af4a5'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:37.239028 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-57091d8e-959c-4b98-9dac-164cd28af4a5 admin admin] The token cache is being invalidate because project be6cb72c4d5b4b63afea584e8e5bb933 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:50:37.243518 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-57091d8e-959c-4b98-9dac-164cd28af4a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:37.244291 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-57091d8e-959c-4b98-9dac-164cd28af4a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:37.247481 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1056/2111] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:50:37 2022] DELETE /identity/v3/projects/be6cb72c4d5b4b63afea584e8e5bb933 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:50:37.251572 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141fc0d-9576-4823-a211-abc78731472f service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:37.261439 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141fc0d-9576-4823-a211-abc78731472f service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:37.313181 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:37.338200 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69443ff7-5939-40d8-98c3-8cc81d3a98bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:37.339119 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69443ff7-5939-40d8-98c3-8cc81d3a98bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:37.339412 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69443ff7-5939-40d8-98c3-8cc81d3a98bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:37.339695 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69443ff7-5939-40d8-98c3-8cc81d3a98bd admin admin] PATH_INFO: `/v3/users/25c561ce2ff44765b85dda08b73a6cc8` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:37.340918 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69443ff7-5939-40d8-98c3-8cc81d3a98bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=25c561ce2ff44765b85dda08b73a6cc8)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:37.342866 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69443ff7-5939-40d8-98c3-8cc81d3a98bd admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:37.394777 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-69443ff7-5939-40d8-98c3-8cc81d3a98bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25c561ce2ff44765b85dda08b73a6cc8'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:37.407532 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-69443ff7-5939-40d8-98c3-8cc81d3a98bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25c561ce2ff44765b85dda08b73a6cc8'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:37.408402 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-69443ff7-5939-40d8-98c3-8cc81d3a98bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25c561ce2ff44765b85dda08b73a6cc8'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:37.409288 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-69443ff7-5939-40d8-98c3-8cc81d3a98bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25c561ce2ff44765b85dda08b73a6cc8'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:37.417286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-69443ff7-5939-40d8-98c3-8cc81d3a98bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25c561ce2ff44765b85dda08b73a6cc8'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:37.448527 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1056/2112] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:50:37 2022] DELETE /identity/v3/users/25c561ce2ff44765b85dda08b73a6cc8 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:50:37.795962 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57091d8e-959c-4b98-9dac-164cd28af4a5 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:37.798341 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-57091d8e-959c-4b98-9dac-164cd28af4a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:37.813181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57091d8e-959c-4b98-9dac-164cd28af4a5 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:37.877600 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c279e79f-0018-4ef8-99b5-4dcaf9bebffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:37.878528 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279e79f-0018-4ef8-99b5-4dcaf9bebffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:37.878807 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279e79f-0018-4ef8-99b5-4dcaf9bebffd admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:37.879117 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279e79f-0018-4ef8-99b5-4dcaf9bebffd admin admin] PATH_INFO: `/v3/projects/f3080730558f4971acb5eeab62199441` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:37.880336 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c279e79f-0018-4ef8-99b5-4dcaf9bebffd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3080730558f4971acb5eeab62199441)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:37.882274 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c279e79f-0018-4ef8-99b5-4dcaf9bebffd admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:37.930884 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-c279e79f-0018-4ef8-99b5-4dcaf9bebffd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3080730558f4971acb5eeab62199441', 'request_id': 'req-c279e79f-0018-4ef8-99b5-4dcaf9bebffd'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:37.936929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-c279e79f-0018-4ef8-99b5-4dcaf9bebffd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3080730558f4971acb5eeab62199441', 'request_id': 'req-c279e79f-0018-4ef8-99b5-4dcaf9bebffd'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:37.946557 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-c279e79f-0018-4ef8-99b5-4dcaf9bebffd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3080730558f4971acb5eeab62199441', 'request_id': 'req-c279e79f-0018-4ef8-99b5-4dcaf9bebffd'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:37.962984 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-c279e79f-0018-4ef8-99b5-4dcaf9bebffd admin admin] The token cache is being invalidate because project f3080730558f4971acb5eeab62199441 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:50:37.963470 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-c279e79f-0018-4ef8-99b5-4dcaf9bebffd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:37.964496 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-c279e79f-0018-4ef8-99b5-4dcaf9bebffd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:37.966885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1057/2113] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:50:37 2022] DELETE /identity/v3/projects/f3080730558f4971acb5eeab62199441 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:50:37.977397 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afab267-cba0-4d51-b3b8-8ead65bb05a5 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:37.977666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afab267-cba0-4d51-b3b8-8ead65bb05a5 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:37.978093 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afab267-cba0-4d51-b3b8-8ead65bb05a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:38.000438 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-5afab267-cba0-4d51-b3b8-8ead65bb05a5 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:38.001672 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-5afab267-cba0-4d51-b3b8-8ead65bb05a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:38.081896 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1057/2114] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:50:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:38.092454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80629926-794f-4703-a978-bae4bdf3e2bc None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:38.093032 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80629926-794f-4703-a978-bae4bdf3e2bc None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:38.093235 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80629926-794f-4703-a978-bae4bdf3e2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:38.113107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-80629926-794f-4703-a978-bae4bdf3e2bc None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:38.114591 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-80629926-794f-4703-a978-bae4bdf3e2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:38.123095 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1058/2115] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:50:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:38.127601 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afab267-cba0-4d51-b3b8-8ead65bb05a5 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:38.137638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afab267-cba0-4d51-b3b8-8ead65bb05a5 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:38.143402 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca88459-9033-4e30-a04e-56e8ab10d8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:38.144186 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca88459-9033-4e30-a04e-56e8ab10d8d1 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:38.144550 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca88459-9033-4e30-a04e-56e8ab10d8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:38.144830 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca88459-9033-4e30-a04e-56e8ab10d8d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:38.145831 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca88459-9033-4e30-a04e-56e8ab10d8d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:50:38.146136 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca88459-9033-4e30-a04e-56e8ab10d8d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:38.148054 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca88459-9033-4e30-a04e-56e8ab10d8d1 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:38.161386 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1058/2116] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:50:38.166985 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80629926-794f-4703-a978-bae4bdf3e2bc None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:38.176987 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80629926-794f-4703-a978-bae4bdf3e2bc None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:38.182407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890275f-6a89-4836-a3aa-d62d00f9db71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:38.183007 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890275f-6a89-4836-a3aa-d62d00f9db71 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:38.183295 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890275f-6a89-4836-a3aa-d62d00f9db71 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:38.183571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890275f-6a89-4836-a3aa-d62d00f9db71 admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:38.184283 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890275f-6a89-4836-a3aa-d62d00f9db71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:38.186204 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890275f-6a89-4836-a3aa-d62d00f9db71 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:38.201465 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:38.225851 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1059/2117] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:50:38 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:50:38.227021 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca88459-9033-4e30-a04e-56e8ab10d8d1 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:38.243960 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca88459-9033-4e30-a04e-56e8ab10d8d1 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:38.300316 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:38.315814 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba638eb-f561-4644-a5e1-f8a47b695d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:38.316519 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba638eb-f561-4644-a5e1-f8a47b695d17 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:38.316784 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba638eb-f561-4644-a5e1-f8a47b695d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:38.317058 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba638eb-f561-4644-a5e1-f8a47b695d17 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:38.317785 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba638eb-f561-4644-a5e1-f8a47b695d17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:38.319825 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba638eb-f561-4644-a5e1-f8a47b695d17 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:38.344426 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1059/2118] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:50:38 2022] POST /identity/v3/users => generated 292 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:50:38.345882 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890275f-6a89-4836-a3aa-d62d00f9db71 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:38.360522 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890275f-6a89-4836-a3aa-d62d00f9db71 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:38.365393 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4a5bf-c6e0-4ee4-b39e-33b4034e8e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:38.366071 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4a5bf-c6e0-4ee4-b39e-33b4034e8e5d admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:38.366350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4a5bf-c6e0-4ee4-b39e-33b4034e8e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:38.366621 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4a5bf-c6e0-4ee4-b39e-33b4034e8e5d admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:38.367390 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c4a5bf-c6e0-4ee4-b39e-33b4034e8e5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:38.369009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c4a5bf-c6e0-4ee4-b39e-33b4034e8e5d admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:38.381241 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1060/2119] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:50:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:50:38.385350 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba638eb-f561-4644-a5e1-f8a47b695d17 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:38.396168 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba638eb-f561-4644-a5e1-f8a47b695d17 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:38.401137 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd33ae-bd31-4057-b661-163c6f4d7805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:38.401820 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd33ae-bd31-4057-b661-163c6f4d7805 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:38.402125 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd33ae-bd31-4057-b661-163c6f4d7805 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:38.402397 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd33ae-bd31-4057-b661-163c6f4d7805 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:38.403160 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdd33ae-bd31-4057-b661-163c6f4d7805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:38.405401 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdd33ae-bd31-4057-b661-163c6f4d7805 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:38.417042 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1060/2120] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:50:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:50:38.421445 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4a5bf-c6e0-4ee4-b39e-33b4034e8e5d admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:38.432244 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4a5bf-c6e0-4ee4-b39e-33b4034e8e5d admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:38.438363 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d03eb-b42a-4214-b2b0-d59a9706746f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:38.439240 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d03eb-b42a-4214-b2b0-d59a9706746f admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:38.439607 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d03eb-b42a-4214-b2b0-d59a9706746f admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:38.439884 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d03eb-b42a-4214-b2b0-d59a9706746f admin admin] PATH_INFO: `/v3/projects/21e797507bf24d9385ad82a74030e2f3/users/c2673167c5ba4603bbe5dcc5f9ad488a/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:38.456551 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528d03eb-b42a-4214-b2b0-d59a9706746f admin admin] RBAC: Authorizing `identity:create_grant(project_id=21e797507bf24d9385ad82a74030e2f3, user_id=c2673167c5ba4603bbe5dcc5f9ad488a, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:38.459295 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528d03eb-b42a-4214-b2b0-d59a9706746f admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:38.474050 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1061/2121] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:50:38 2022] PUT /identity/v3/projects/21e797507bf24d9385ad82a74030e2f3/users/c2673167c5ba4603bbe5dcc5f9ad488a/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:50:38.485707 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe464a-e262-4bde-a57c-e0be33a358af None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:38.486028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe464a-e262-4bde-a57c-e0be33a358af None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:38.486431 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe464a-e262-4bde-a57c-e0be33a358af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:38.515622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-a6fe464a-e262-4bde-a57c-e0be33a358af None None] MFA Rules not processed for user `c2673167c5ba4603bbe5dcc5f9ad488a`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:38.516877 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-a6fe464a-e262-4bde-a57c-e0be33a358af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:38.589386 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1061/2122] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:50:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:38.600891 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d03eb-b42a-4214-b2b0-d59a9706746f admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:38.602861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-528d03eb-b42a-4214-b2b0-d59a9706746f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:38.663130 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d03eb-b42a-4214-b2b0-d59a9706746f admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:38.667243 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3014f-3ddf-494d-9bd2-def5b6e2ccc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:38.667887 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3014f-3ddf-494d-9bd2-def5b6e2ccc0 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:38.668191 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3014f-3ddf-494d-9bd2-def5b6e2ccc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:38.668416 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3014f-3ddf-494d-9bd2-def5b6e2ccc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:38.678378 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa3014f-3ddf-494d-9bd2-def5b6e2ccc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:38.679856 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa3014f-3ddf-494d-9bd2-def5b6e2ccc0 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:38.690837 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1062/2123] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:50:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:50:38.877723 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a2818-02ba-44fc-9151-eacf64315d5f None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:38.878026 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a2818-02ba-44fc-9151-eacf64315d5f None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:38.878314 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a2818-02ba-44fc-9151-eacf64315d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:38.898785 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-860a2818-02ba-44fc-9151-eacf64315d5f None None] MFA Rules not processed for user `2f8cf6d908314281a3ddc9d9513d39c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:38.900133 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-860a2818-02ba-44fc-9151-eacf64315d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:38.971866 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1062/2124] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Jul 20 14:50:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:39.024607 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3014f-3ddf-494d-9bd2-def5b6e2ccc0 service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:39.026335 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-0aa3014f-3ddf-494d-9bd2-def5b6e2ccc0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:39.100449 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3014f-3ddf-494d-9bd2-def5b6e2ccc0 service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:39.105636 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd0749-951c-459c-ba1a-0ac5c38831c7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f4844988474d7a91133d77906611a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:39.106291 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd0749-951c-459c-ba1a-0ac5c38831c7 service swift] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:39.106571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd0749-951c-459c-ba1a-0ac5c38831c7 service swift] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:39.106844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd0749-951c-459c-ba1a-0ac5c38831c7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:39.116318 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfd0749-951c-459c-ba1a-0ac5c38831c7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:39.118079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfd0749-951c-459c-ba1a-0ac5c38831c7 service swift] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:39.127366 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1063/2125] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jul 20 14:50:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 14:50:39.749647 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22975e98-7549-4ec3-b71d-e6b71f97087b None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:39.750120 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22975e98-7549-4ec3-b71d-e6b71f97087b None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:39.750233 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22975e98-7549-4ec3-b71d-e6b71f97087b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:39.770081 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-22975e98-7549-4ec3-b71d-e6b71f97087b None None] MFA Rules not processed for user `2f8cf6d908314281a3ddc9d9513d39c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:39.771270 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-22975e98-7549-4ec3-b71d-e6b71f97087b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:39.780622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1063/2126] 10.222.0.15 () {58 vars in 967 bytes} [Wed Jul 20 14:50:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:39.794564 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd0749-951c-459c-ba1a-0ac5c38831c7 service swift] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:39.810263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd0749-951c-459c-ba1a-0ac5c38831c7 service swift] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:39.818661 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc31f433-d37b-4184-be29-de60c583ac5b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f4844988474d7a91133d77906611a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:39.819320 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31f433-d37b-4184-be29-de60c583ac5b service swift] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:39.819603 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31f433-d37b-4184-be29-de60c583ac5b service swift] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:39.819877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31f433-d37b-4184-be29-de60c583ac5b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:39.829631 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc31f433-d37b-4184-be29-de60c583ac5b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:39.831312 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc31f433-d37b-4184-be29-de60c583ac5b service swift] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:39.839881 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1064/2127] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jul 20 14:50:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 14:50:40.079744 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142690ce-ae63-4055-bc95-20500a1e7255 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:40.080050 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142690ce-ae63-4055-bc95-20500a1e7255 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:40.080341 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142690ce-ae63-4055-bc95-20500a1e7255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:40.104352 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-142690ce-ae63-4055-bc95-20500a1e7255 None None] MFA Rules not processed for user `2f8cf6d908314281a3ddc9d9513d39c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:40.105652 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-142690ce-ae63-4055-bc95-20500a1e7255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:40.116082 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1064/2128] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Jul 20 14:50:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:40.130084 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc31f433-d37b-4184-be29-de60c583ac5b service swift] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:40.140893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc31f433-d37b-4184-be29-de60c583ac5b service swift] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:40.146439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f5af73-326d-4f67-b791-9de6d1f71634 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f4844988474d7a91133d77906611a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:40.147136 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5af73-326d-4f67-b791-9de6d1f71634 service swift] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:40.147426 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5af73-326d-4f67-b791-9de6d1f71634 service swift] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:40.147759 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5af73-326d-4f67-b791-9de6d1f71634 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:40.157532 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f5af73-326d-4f67-b791-9de6d1f71634 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:40.159231 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f5af73-326d-4f67-b791-9de6d1f71634 service swift] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:40.168645 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1065/2129] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jul 20 14:50:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 14:50:40.534406 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67e4f9-1466-44c3-b943-93593d8c8cd1 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:40.534633 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67e4f9-1466-44c3-b943-93593d8c8cd1 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:40.534917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67e4f9-1466-44c3-b943-93593d8c8cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:40.555853 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-9b67e4f9-1466-44c3-b943-93593d8c8cd1 None None] MFA Rules not processed for user `2f8cf6d908314281a3ddc9d9513d39c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:40.557130 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-9b67e4f9-1466-44c3-b943-93593d8c8cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:40.565887 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1065/2130] 10.222.0.15 () {58 vars in 967 bytes} [Wed Jul 20 14:50:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:40.585576 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f5af73-326d-4f67-b791-9de6d1f71634 service swift] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:40.598895 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f5af73-326d-4f67-b791-9de6d1f71634 service swift] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:40.604163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34513658-1620-4a4a-844a-e9cc9ab0c29b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f4844988474d7a91133d77906611a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:40.604819 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34513658-1620-4a4a-844a-e9cc9ab0c29b service swift] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:40.605095 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34513658-1620-4a4a-844a-e9cc9ab0c29b service swift] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:40.605387 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34513658-1620-4a4a-844a-e9cc9ab0c29b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:40.614119 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34513658-1620-4a4a-844a-e9cc9ab0c29b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:40.615741 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34513658-1620-4a4a-844a-e9cc9ab0c29b service swift] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:40.624504 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1066/2131] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jul 20 14:50:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 14:50:50.175266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b67e4f9-1466-44c3-b943-93593d8c8cd1 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:50.176868 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-9b67e4f9-1466-44c3-b943-93593d8c8cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:50.232481 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:50.263283 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b67e4f9-1466-44c3-b943-93593d8c8cd1 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:50.268249 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa235a5-a125-47a5-a38c-428a6e6a23cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:50.269163 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa235a5-a125-47a5-a38c-428a6e6a23cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:50.269448 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa235a5-a125-47a5-a38c-428a6e6a23cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:50.269783 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa235a5-a125-47a5-a38c-428a6e6a23cc admin admin] PATH_INFO: `/v3/users/e1750ad6625e4cc0b430a81ed188e538` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:50.271101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa235a5-a125-47a5-a38c-428a6e6a23cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1750ad6625e4cc0b430a81ed188e538)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:50.273083 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa235a5-a125-47a5-a38c-428a6e6a23cc admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:50.326593 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-3fa235a5-a125-47a5-a38c-428a6e6a23cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1750ad6625e4cc0b430a81ed188e538'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:50.338923 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-3fa235a5-a125-47a5-a38c-428a6e6a23cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1750ad6625e4cc0b430a81ed188e538'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:50.339659 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-3fa235a5-a125-47a5-a38c-428a6e6a23cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1750ad6625e4cc0b430a81ed188e538'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:50.340275 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-3fa235a5-a125-47a5-a38c-428a6e6a23cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1750ad6625e4cc0b430a81ed188e538'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:50.347239 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-3fa235a5-a125-47a5-a38c-428a6e6a23cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1750ad6625e4cc0b430a81ed188e538'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:50.361462 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1066/2132] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:50:50 2022] DELETE /identity/v3/users/e1750ad6625e4cc0b430a81ed188e538 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:50:50.366409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabf89b-7341-47f5-aef9-e12fcb675d59 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:50.366944 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabf89b-7341-47f5-aef9-e12fcb675d59 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:50.367039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabf89b-7341-47f5-aef9-e12fcb675d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:50.385747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-bcabf89b-7341-47f5-aef9-e12fcb675d59 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:50.386857 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-bcabf89b-7341-47f5-aef9-e12fcb675d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:50.396115 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:50.450986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1067/2133] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:50:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:50.461088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa235a5-a125-47a5-a38c-428a6e6a23cc admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:50.462595 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-3fa235a5-a125-47a5-a38c-428a6e6a23cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:50.536667 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa235a5-a125-47a5-a38c-428a6e6a23cc admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:50.540572 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f51a0-734c-46c0-9e50-9113642d1286 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:50.541259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f51a0-734c-46c0-9e50-9113642d1286 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:50.541548 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f51a0-734c-46c0-9e50-9113642d1286 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:50.541827 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f51a0-734c-46c0-9e50-9113642d1286 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:50.553305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177f51a0-734c-46c0-9e50-9113642d1286 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:50.556266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177f51a0-734c-46c0-9e50-9113642d1286 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:50.568243 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1067/2134] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:50:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:50:50.751262 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcabf89b-7341-47f5-aef9-e12fcb675d59 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:50.752865 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-bcabf89b-7341-47f5-aef9-e12fcb675d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:50.770271 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcabf89b-7341-47f5-aef9-e12fcb675d59 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:50.780498 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c1d7f-4fd9-4486-9479-149540669c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:50.782641 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c1d7f-4fd9-4486-9479-149540669c9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:50.783805 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c1d7f-4fd9-4486-9479-149540669c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:50.784043 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c1d7f-4fd9-4486-9479-149540669c9b admin admin] PATH_INFO: `/v3/projects/50a5f7380c804eba98ab4a8ee65dc1d3` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:50.786102 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2c1d7f-4fd9-4486-9479-149540669c9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=50a5f7380c804eba98ab4a8ee65dc1d3)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:50.788089 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2c1d7f-4fd9-4486-9479-149540669c9b admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:50.846227 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1c2c1d7f-4fd9-4486-9479-149540669c9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50a5f7380c804eba98ab4a8ee65dc1d3', 'request_id': 'req-1c2c1d7f-4fd9-4486-9479-149540669c9b'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:50.852005 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1c2c1d7f-4fd9-4486-9479-149540669c9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50a5f7380c804eba98ab4a8ee65dc1d3', 'request_id': 'req-1c2c1d7f-4fd9-4486-9479-149540669c9b'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:50.860627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1c2c1d7f-4fd9-4486-9479-149540669c9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50a5f7380c804eba98ab4a8ee65dc1d3', 'request_id': 'req-1c2c1d7f-4fd9-4486-9479-149540669c9b'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:50.874258 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1c2c1d7f-4fd9-4486-9479-149540669c9b admin admin] The token cache is being invalidate because project 50a5f7380c804eba98ab4a8ee65dc1d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:50:50.874561 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1c2c1d7f-4fd9-4486-9479-149540669c9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:50.875242 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1c2c1d7f-4fd9-4486-9479-149540669c9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:50.876876 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1068/2135] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:50:50 2022] DELETE /identity/v3/projects/50a5f7380c804eba98ab4a8ee65dc1d3 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:50:51.276306 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f51a0-734c-46c0-9e50-9113642d1286 service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:51.278829 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-177f51a0-734c-46c0-9e50-9113642d1286 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:51.364809 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f51a0-734c-46c0-9e50-9113642d1286 service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:51.369498 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:51.432008 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5adf4-f18b-477c-a9a0-da6ad8b2e561 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb857f32bbc04d82bfa681bd5659765f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:51.432789 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5adf4-f18b-477c-a9a0-da6ad8b2e561 service placement] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:51.433079 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5adf4-f18b-477c-a9a0-da6ad8b2e561 service placement] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:51.433369 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5adf4-f18b-477c-a9a0-da6ad8b2e561 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:51.435301 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-9ff5adf4-f18b-477c-a9a0-da6ad8b2e561 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:51.447545 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff5adf4-f18b-477c-a9a0-da6ad8b2e561 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:51.449159 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff5adf4-f18b-477c-a9a0-da6ad8b2e561 service placement] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:51.459890 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1068/2136] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:50:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:50:52.385540 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c1d7f-4fd9-4486-9479-149540669c9b admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:52.387718 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-1c2c1d7f-4fd9-4486-9479-149540669c9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:52.458322 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c1d7f-4fd9-4486-9479-149540669c9b admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:52.467228 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6bbac5-8d4b-4d0f-b770-ab2d16372192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:52.468622 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bbac5-8d4b-4d0f-b770-ab2d16372192 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:52.469099 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bbac5-8d4b-4d0f-b770-ab2d16372192 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:52.469562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bbac5-8d4b-4d0f-b770-ab2d16372192 admin admin] PATH_INFO: `/v3/users/c2673167c5ba4603bbe5dcc5f9ad488a` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:52.471833 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6bbac5-8d4b-4d0f-b770-ab2d16372192 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2673167c5ba4603bbe5dcc5f9ad488a)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:52.475340 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6bbac5-8d4b-4d0f-b770-ab2d16372192 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:52.529523 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:52.536691 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-dc6bbac5-8d4b-4d0f-b770-ab2d16372192 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2673167c5ba4603bbe5dcc5f9ad488a'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:52.550068 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-dc6bbac5-8d4b-4d0f-b770-ab2d16372192 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2673167c5ba4603bbe5dcc5f9ad488a'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:52.550876 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-dc6bbac5-8d4b-4d0f-b770-ab2d16372192 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2673167c5ba4603bbe5dcc5f9ad488a'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:52.555200 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-dc6bbac5-8d4b-4d0f-b770-ab2d16372192 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2673167c5ba4603bbe5dcc5f9ad488a'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:52.564833 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-dc6bbac5-8d4b-4d0f-b770-ab2d16372192 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2673167c5ba4603bbe5dcc5f9ad488a'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:52.580496 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1069/2137] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:50:52 2022] DELETE /identity/v3/users/c2673167c5ba4603bbe5dcc5f9ad488a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:50:52.585480 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63f500-d8e5-42ff-90bd-0a104212f9f4 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:52.585955 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63f500-d8e5-42ff-90bd-0a104212f9f4 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:52.586047 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63f500-d8e5-42ff-90bd-0a104212f9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:52.604786 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-bb63f500-d8e5-42ff-90bd-0a104212f9f4 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:52.606051 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-bb63f500-d8e5-42ff-90bd-0a104212f9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:52.668957 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1069/2138] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:50:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:50:52.678978 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6bbac5-8d4b-4d0f-b770-ab2d16372192 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:52.681941 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-dc6bbac5-8d4b-4d0f-b770-ab2d16372192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:52.743847 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6bbac5-8d4b-4d0f-b770-ab2d16372192 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:52.748413 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6583cb2a-d5b8-41cc-a441-715cd1513571 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:52.749167 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583cb2a-d5b8-41cc-a441-715cd1513571 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:52.749343 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583cb2a-d5b8-41cc-a441-715cd1513571 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:52.749647 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583cb2a-d5b8-41cc-a441-715cd1513571 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:52.759983 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6583cb2a-d5b8-41cc-a441-715cd1513571 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:52.761418 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6583cb2a-d5b8-41cc-a441-715cd1513571 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:52.771758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1070/2139] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:50:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:50:53.055942 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63f500-d8e5-42ff-90bd-0a104212f9f4 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:53.057517 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-bb63f500-d8e5-42ff-90bd-0a104212f9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:53.070150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63f500-d8e5-42ff-90bd-0a104212f9f4 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:53.075342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5bd0b1-f093-44cd-b6bc-35daca5d923b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:53.076269 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5bd0b1-f093-44cd-b6bc-35daca5d923b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:53.076549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5bd0b1-f093-44cd-b6bc-35daca5d923b admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:53.076831 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5bd0b1-f093-44cd-b6bc-35daca5d923b admin admin] PATH_INFO: `/v3/projects/21e797507bf24d9385ad82a74030e2f3` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:53.078077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5bd0b1-f093-44cd-b6bc-35daca5d923b admin admin] RBAC: Authorizing `identity:delete_project(project_id=21e797507bf24d9385ad82a74030e2f3)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:53.080152 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5bd0b1-f093-44cd-b6bc-35daca5d923b admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:53.129303 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ba5bd0b1-f093-44cd-b6bc-35daca5d923b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21e797507bf24d9385ad82a74030e2f3', 'request_id': 'req-ba5bd0b1-f093-44cd-b6bc-35daca5d923b'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:53.134933 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ba5bd0b1-f093-44cd-b6bc-35daca5d923b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21e797507bf24d9385ad82a74030e2f3', 'request_id': 'req-ba5bd0b1-f093-44cd-b6bc-35daca5d923b'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:53.149892 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ba5bd0b1-f093-44cd-b6bc-35daca5d923b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21e797507bf24d9385ad82a74030e2f3', 'request_id': 'req-ba5bd0b1-f093-44cd-b6bc-35daca5d923b'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:53.160303 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ba5bd0b1-f093-44cd-b6bc-35daca5d923b admin admin] The token cache is being invalidate because project 21e797507bf24d9385ad82a74030e2f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:50:53.160616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ba5bd0b1-f093-44cd-b6bc-35daca5d923b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:53.161317 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ba5bd0b1-f093-44cd-b6bc-35daca5d923b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:53.162929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1070/2140] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:50:53 2022] DELETE /identity/v3/projects/21e797507bf24d9385ad82a74030e2f3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:50:53.167972 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b099d-d7a4-4c3b-85ef-f562d8b28109 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:53.168272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b099d-d7a4-4c3b-85ef-f562d8b28109 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:53.168581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b099d-d7a4-4c3b-85ef-f562d8b28109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:53.188353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-e14b099d-d7a4-4c3b-85ef-f562d8b28109 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:53.189706 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-e14b099d-d7a4-4c3b-85ef-f562d8b28109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:53.255587 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1071/2141] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:50:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:53.264538 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43dd1aa-e64d-4438-ba8c-468ce2046d0a None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:53.264830 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43dd1aa-e64d-4438-ba8c-468ce2046d0a None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:53.265110 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43dd1aa-e64d-4438-ba8c-468ce2046d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:53.284872 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-b43dd1aa-e64d-4438-ba8c-468ce2046d0a None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:53.286336 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-b43dd1aa-e64d-4438-ba8c-468ce2046d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:53.295081 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1071/2142] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:50:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:53.300107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b099d-d7a4-4c3b-85ef-f562d8b28109 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:53.316641 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b099d-d7a4-4c3b-85ef-f562d8b28109 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:53.322126 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831baed-dc45-42b2-b86d-bd169ba4493a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:53.322802 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831baed-dc45-42b2-b86d-bd169ba4493a admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:53.323112 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831baed-dc45-42b2-b86d-bd169ba4493a admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:53.323411 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831baed-dc45-42b2-b86d-bd169ba4493a admin admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:53.324134 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c831baed-dc45-42b2-b86d-bd169ba4493a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:50:53.324451 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c831baed-dc45-42b2-b86d-bd169ba4493a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:53.326103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c831baed-dc45-42b2-b86d-bd169ba4493a admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:53.339101 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1072/2143] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:50:53.344305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43dd1aa-e64d-4438-ba8c-468ce2046d0a None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:53.359011 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43dd1aa-e64d-4438-ba8c-468ce2046d0a None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:53.365622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5558f74e-6a41-4c5a-b6e5-920358e6a9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:53.366421 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5558f74e-6a41-4c5a-b6e5-920358e6a9b8 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:53.366713 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5558f74e-6a41-4c5a-b6e5-920358e6a9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:53.366994 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5558f74e-6a41-4c5a-b6e5-920358e6a9b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:53.367755 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5558f74e-6a41-4c5a-b6e5-920358e6a9b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:53.369720 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5558f74e-6a41-4c5a-b6e5-920358e6a9b8 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:53.385188 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:53.402347 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1072/2144] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:50:53 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:50:53.407006 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831baed-dc45-42b2-b86d-bd169ba4493a admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:53.417652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831baed-dc45-42b2-b86d-bd169ba4493a admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:53.465321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:53.480680 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f928cd6-d190-4a5d-b837-a3597e000906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:53.485942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f928cd6-d190-4a5d-b837-a3597e000906 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:53.485942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f928cd6-d190-4a5d-b837-a3597e000906 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:53.485942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f928cd6-d190-4a5d-b837-a3597e000906 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:53.485942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f928cd6-d190-4a5d-b837-a3597e000906 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:53.485942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f928cd6-d190-4a5d-b837-a3597e000906 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:53.506582 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1073/2145] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:50:53 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 14:50:53.510664 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5558f74e-6a41-4c5a-b6e5-920358e6a9b8 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:53.522161 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5558f74e-6a41-4c5a-b6e5-920358e6a9b8 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:53.527417 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd65bc-5bfb-4a2d-b31a-7b40c183288b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:53.528421 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd65bc-5bfb-4a2d-b31a-7b40c183288b admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:53.528701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd65bc-5bfb-4a2d-b31a-7b40c183288b admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:53.528977 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd65bc-5bfb-4a2d-b31a-7b40c183288b admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:53.529711 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd65bc-5bfb-4a2d-b31a-7b40c183288b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:53.531361 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd65bc-5bfb-4a2d-b31a-7b40c183288b admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:53.545182 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1073/2146] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:50:53.550292 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f928cd6-d190-4a5d-b837-a3597e000906 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:53.565468 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f928cd6-d190-4a5d-b837-a3597e000906 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:53.574774 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202dede6-8c0a-4b3f-ad0f-3eb27d1548e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:53.575730 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202dede6-8c0a-4b3f-ad0f-3eb27d1548e8 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:53.576011 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202dede6-8c0a-4b3f-ad0f-3eb27d1548e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:53.576286 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202dede6-8c0a-4b3f-ad0f-3eb27d1548e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:53.577023 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202dede6-8c0a-4b3f-ad0f-3eb27d1548e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:53.578647 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202dede6-8c0a-4b3f-ad0f-3eb27d1548e8 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:53.590585 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1074/2147] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:50:53.594914 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd65bc-5bfb-4a2d-b31a-7b40c183288b admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:53.607906 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd65bc-5bfb-4a2d-b31a-7b40c183288b admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:53.613304 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5973f-9597-4bbc-ba19-b5eda86caabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:53.614364 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5973f-9597-4bbc-ba19-b5eda86caabc admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:53.614740 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5973f-9597-4bbc-ba19-b5eda86caabc admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:53.615018 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5973f-9597-4bbc-ba19-b5eda86caabc admin admin] PATH_INFO: `/v3/projects/e2902ebce68e4526bc764fc1db9cda63/users/7e189c82ba74427cb3908e8294abe64d/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:53.637384 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b5973f-9597-4bbc-ba19-b5eda86caabc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2902ebce68e4526bc764fc1db9cda63, user_id=7e189c82ba74427cb3908e8294abe64d, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:53.640159 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b5973f-9597-4bbc-ba19-b5eda86caabc admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:53.657990 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1074/2148] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:50:53 2022] PUT /identity/v3/projects/e2902ebce68e4526bc764fc1db9cda63/users/7e189c82ba74427cb3908e8294abe64d/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:50:53.668544 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e3fd2-a51d-4c9b-b4b9-0b286f05181f None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:53.668833 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e3fd2-a51d-4c9b-b4b9-0b286f05181f None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:53.669107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e3fd2-a51d-4c9b-b4b9-0b286f05181f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:53.696905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-ce3e3fd2-a51d-4c9b-b4b9-0b286f05181f None None] MFA Rules not processed for user `7e189c82ba74427cb3908e8294abe64d`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:53.698162 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-ce3e3fd2-a51d-4c9b-b4b9-0b286f05181f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:53.772921 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1075/2149] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:50:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:53.777623 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5973f-9597-4bbc-ba19-b5eda86caabc admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:53.845659 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5973f-9597-4bbc-ba19-b5eda86caabc admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:53.850759 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d09410a-f94b-48e5-bfb1-7c930befc272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:53.851459 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d09410a-f94b-48e5-bfb1-7c930befc272 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:53.851738 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d09410a-f94b-48e5-bfb1-7c930befc272 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:53.852025 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d09410a-f94b-48e5-bfb1-7c930befc272 admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:53.852737 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d09410a-f94b-48e5-bfb1-7c930befc272 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:53.854642 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d09410a-f94b-48e5-bfb1-7c930befc272 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:53.882111 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1075/2150] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:50:53 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:50:53.887329 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e3fd2-a51d-4c9b-b4b9-0b286f05181f None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:53.898366 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e3fd2-a51d-4c9b-b4b9-0b286f05181f None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:53.964954 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36761bdf-8b19-478d-ac01-6fb4cc0a33d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:53.965669 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36761bdf-8b19-478d-ac01-6fb4cc0a33d1 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:53.966053 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36761bdf-8b19-478d-ac01-6fb4cc0a33d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:53.966339 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36761bdf-8b19-478d-ac01-6fb4cc0a33d1 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:53.967070 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36761bdf-8b19-478d-ac01-6fb4cc0a33d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:53.969011 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36761bdf-8b19-478d-ac01-6fb4cc0a33d1 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:53.998498 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1076/2151] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:50:53 2022] POST /identity/v3/users => generated 294 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:50:53.999834 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d09410a-f94b-48e5-bfb1-7c930befc272 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:54.011693 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d09410a-f94b-48e5-bfb1-7c930befc272 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:54.018857 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2875452-10cf-4935-bc66-1584e557c4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:54.019583 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2875452-10cf-4935-bc66-1584e557c4ff admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:54.019871 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2875452-10cf-4935-bc66-1584e557c4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:54.020152 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2875452-10cf-4935-bc66-1584e557c4ff admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:54.020905 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2875452-10cf-4935-bc66-1584e557c4ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:54.022992 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2875452-10cf-4935-bc66-1584e557c4ff admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:54.034827 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1076/2152] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:50:54.039565 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36761bdf-8b19-478d-ac01-6fb4cc0a33d1 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:54.049239 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36761bdf-8b19-478d-ac01-6fb4cc0a33d1 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:54.054100 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad57d24-fd5d-4e48-ae72-ddb33f29cd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:54.055009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad57d24-fd5d-4e48-ae72-ddb33f29cd0e admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:54.055368 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad57d24-fd5d-4e48-ae72-ddb33f29cd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:54.055643 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad57d24-fd5d-4e48-ae72-ddb33f29cd0e admin admin] PATH_INFO: `/v3/projects/345a263c03474a16a4a44b14e44e11e9/users/494cf20b9e544302a3b405ca182e5e40/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:54.073951 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad57d24-fd5d-4e48-ae72-ddb33f29cd0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=345a263c03474a16a4a44b14e44e11e9, user_id=494cf20b9e544302a3b405ca182e5e40, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:54.077354 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad57d24-fd5d-4e48-ae72-ddb33f29cd0e admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:54.092261 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1077/2153] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:50:54 2022] PUT /identity/v3/projects/345a263c03474a16a4a44b14e44e11e9/users/494cf20b9e544302a3b405ca182e5e40/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:50:54.096299 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2875452-10cf-4935-bc66-1584e557c4ff admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:54.107087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2875452-10cf-4935-bc66-1584e557c4ff admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:54.175649 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10111f0-24e3-435a-b08f-99c40c355dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:54.176380 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10111f0-24e3-435a-b08f-99c40c355dba admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:54.176605 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10111f0-24e3-435a-b08f-99c40c355dba admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:54.176883 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10111f0-24e3-435a-b08f-99c40c355dba admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:54.177621 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10111f0-24e3-435a-b08f-99c40c355dba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:54.179283 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10111f0-24e3-435a-b08f-99c40c355dba admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:54.189801 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1077/2154] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:50:54.194101 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad57d24-fd5d-4e48-ae72-ddb33f29cd0e admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:54.205788 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad57d24-fd5d-4e48-ae72-ddb33f29cd0e admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:54.211669 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d13cdd-1aaf-4eb3-846b-e605a579ab00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:54.212571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d13cdd-1aaf-4eb3-846b-e605a579ab00 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:54.212856 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d13cdd-1aaf-4eb3-846b-e605a579ab00 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:54.213134 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d13cdd-1aaf-4eb3-846b-e605a579ab00 admin admin] PATH_INFO: `/v3/projects/345a263c03474a16a4a44b14e44e11e9/users/494cf20b9e544302a3b405ca182e5e40/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:54.215453 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d13cdd-1aaf-4eb3-846b-e605a579ab00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=345a263c03474a16a4a44b14e44e11e9, user_id=494cf20b9e544302a3b405ca182e5e40, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:54.218149 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d13cdd-1aaf-4eb3-846b-e605a579ab00 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:54.229810 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1078/2155] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:50:54 2022] PUT /identity/v3/projects/345a263c03474a16a4a44b14e44e11e9/users/494cf20b9e544302a3b405ca182e5e40/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:50:54.241232 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53b007-eab7-4a33-8fb8-142e7a9493e4 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:54.241552 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53b007-eab7-4a33-8fb8-142e7a9493e4 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:54.241885 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53b007-eab7-4a33-8fb8-142e7a9493e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:54.278389 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-3a53b007-eab7-4a33-8fb8-142e7a9493e4 None None] MFA Rules not processed for user `494cf20b9e544302a3b405ca182e5e40`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:54.279589 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-3a53b007-eab7-4a33-8fb8-142e7a9493e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:54.297994 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:54.364034 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1078/2156] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:50:54 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:54.380247 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d13cdd-1aaf-4eb3-846b-e605a579ab00 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:54.382358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-d2d13cdd-1aaf-4eb3-846b-e605a579ab00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:54.393128 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:54.442831 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d13cdd-1aaf-4eb3-846b-e605a579ab00 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:54.446967 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0b1d0-81db-4f51-ad63-34936714d400 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:54.447667 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0b1d0-81db-4f51-ad63-34936714d400 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:54.447976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0b1d0-81db-4f51-ad63-34936714d400 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:54.448235 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0b1d0-81db-4f51-ad63-34936714d400 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:54.509847 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf0b1d0-81db-4f51-ad63-34936714d400 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:54.511425 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf0b1d0-81db-4f51-ad63-34936714d400 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:54.520798 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1079/2157] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:50:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:50:59.169308 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53b007-eab7-4a33-8fb8-142e7a9493e4 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:59.270182 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53b007-eab7-4a33-8fb8-142e7a9493e4 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:59.274881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f99332-07ee-46ad-b1f7-d09de7f8b5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:59.275843 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f99332-07ee-46ad-b1f7-d09de7f8b5d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:59.276161 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f99332-07ee-46ad-b1f7-d09de7f8b5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:59.276443 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f99332-07ee-46ad-b1f7-d09de7f8b5d5 admin admin] PATH_INFO: `/v3/users/7e189c82ba74427cb3908e8294abe64d` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:59.277645 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f99332-07ee-46ad-b1f7-d09de7f8b5d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e189c82ba74427cb3908e8294abe64d)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:59.280102 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f99332-07ee-46ad-b1f7-d09de7f8b5d5 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:59.339804 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-62f99332-07ee-46ad-b1f7-d09de7f8b5d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e189c82ba74427cb3908e8294abe64d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:59.357535 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-62f99332-07ee-46ad-b1f7-d09de7f8b5d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e189c82ba74427cb3908e8294abe64d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:59.358325 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-62f99332-07ee-46ad-b1f7-d09de7f8b5d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e189c82ba74427cb3908e8294abe64d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:59.358769 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-62f99332-07ee-46ad-b1f7-d09de7f8b5d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e189c82ba74427cb3908e8294abe64d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:59.367861 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-62f99332-07ee-46ad-b1f7-d09de7f8b5d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e189c82ba74427cb3908e8294abe64d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:50:59.390291 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1079/2158] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:50:59 2022] DELETE /identity/v3/users/7e189c82ba74427cb3908e8294abe64d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 14:50:59.392279 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a4811b-a261-463e-8f03-cc4eb700f8fd None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:59.392811 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a4811b-a261-463e-8f03-cc4eb700f8fd None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:59.392905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a4811b-a261-463e-8f03-cc4eb700f8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:59.411644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-b5a4811b-a261-463e-8f03-cc4eb700f8fd None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:50:59.413222 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-b5a4811b-a261-463e-8f03-cc4eb700f8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:59.501242 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1080/2159] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:50:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:50:59.512506 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f99332-07ee-46ad-b1f7-d09de7f8b5d5 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:59.514482 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-62f99332-07ee-46ad-b1f7-d09de7f8b5d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:59.578194 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:59.606522 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f99332-07ee-46ad-b1f7-d09de7f8b5d5 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:59.610439 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5caa0-ee2e-4030-8df2-a348fe3a9327 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:59.611269 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5caa0-ee2e-4030-8df2-a348fe3a9327 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:59.611550 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5caa0-ee2e-4030-8df2-a348fe3a9327 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:59.611824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5caa0-ee2e-4030-8df2-a348fe3a9327 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:59.622429 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b5caa0-ee2e-4030-8df2-a348fe3a9327 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:59.623940 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b5caa0-ee2e-4030-8df2-a348fe3a9327 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:50:59.634433 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1080/2160] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:50:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:50:59.940438 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a4811b-a261-463e-8f03-cc4eb700f8fd None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:50:59.942384 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-b5a4811b-a261-463e-8f03-cc4eb700f8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:50:59.959843 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a4811b-a261-463e-8f03-cc4eb700f8fd None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:50:59.965667 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7e7d2-e677-4670-be31-38b5edfc0384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:50:59.966660 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7e7d2-e677-4670-be31-38b5edfc0384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:50:59.966954 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7e7d2-e677-4670-be31-38b5edfc0384 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:50:59.967259 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7e7d2-e677-4670-be31-38b5edfc0384 admin admin] PATH_INFO: `/v3/projects/e2902ebce68e4526bc764fc1db9cda63` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:50:59.968526 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d7e7d2-e677-4670-be31-38b5edfc0384 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2902ebce68e4526bc764fc1db9cda63)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:50:59.970784 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d7e7d2-e677-4670-be31-38b5edfc0384 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:00.038543 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-85d7e7d2-e677-4670-be31-38b5edfc0384 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2902ebce68e4526bc764fc1db9cda63', 'request_id': 'req-85d7e7d2-e677-4670-be31-38b5edfc0384'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:00.044091 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:51:00.045717 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-85d7e7d2-e677-4670-be31-38b5edfc0384 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2902ebce68e4526bc764fc1db9cda63', 'request_id': 'req-85d7e7d2-e677-4670-be31-38b5edfc0384'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:00.067761 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-85d7e7d2-e677-4670-be31-38b5edfc0384 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2902ebce68e4526bc764fc1db9cda63', 'request_id': 'req-85d7e7d2-e677-4670-be31-38b5edfc0384'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:00.084389 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-85d7e7d2-e677-4670-be31-38b5edfc0384 admin admin] The token cache is being invalidate because project e2902ebce68e4526bc764fc1db9cda63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:51:00.084389 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-85d7e7d2-e677-4670-be31-38b5edfc0384 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:00.085183 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-85d7e7d2-e677-4670-be31-38b5edfc0384 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:00.086955 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1081/2161] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:50:59 2022] DELETE /identity/v3/projects/e2902ebce68e4526bc764fc1db9cda63 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:51:00.091401 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5caa0-ee2e-4030-8df2-a348fe3a9327 service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:00.102206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5caa0-ee2e-4030-8df2-a348fe3a9327 service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:00.167316 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b95a0-9fb7-44d2-ac6a-c948d126e914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:00.168210 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b95a0-9fb7-44d2-ac6a-c948d126e914 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:00.168515 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b95a0-9fb7-44d2-ac6a-c948d126e914 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:00.168813 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b95a0-9fb7-44d2-ac6a-c948d126e914 admin admin] PATH_INFO: `/v3/users/494cf20b9e544302a3b405ca182e5e40` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:00.170604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b95a0-9fb7-44d2-ac6a-c948d126e914 admin admin] RBAC: Authorizing `identity:delete_user(user_id=494cf20b9e544302a3b405ca182e5e40)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:00.173944 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b95a0-9fb7-44d2-ac6a-c948d126e914 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:00.221429 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2070b6-2bb8-4e45-bd86-ebbc17c5aa2c None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:00.221683 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2070b6-2bb8-4e45-bd86-ebbc17c5aa2c None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:00.222152 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2070b6-2bb8-4e45-bd86-ebbc17c5aa2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:00.237926 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-0e9b95a0-9fb7-44d2-ac6a-c948d126e914 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '494cf20b9e544302a3b405ca182e5e40'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:00.243141 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-8d2070b6-2bb8-4e45-bd86-ebbc17c5aa2c None None] MFA Rules not processed for user `938083ff74d84224be142a26d8e17ad6`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:51:00.244381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-8d2070b6-2bb8-4e45-bd86-ebbc17c5aa2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:51:00.252254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-0e9b95a0-9fb7-44d2-ac6a-c948d126e914 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '494cf20b9e544302a3b405ca182e5e40'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:00.252980 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-0e9b95a0-9fb7-44d2-ac6a-c948d126e914 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '494cf20b9e544302a3b405ca182e5e40'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:00.255079 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-0e9b95a0-9fb7-44d2-ac6a-c948d126e914 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '494cf20b9e544302a3b405ca182e5e40'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:00.266173 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-0e9b95a0-9fb7-44d2-ac6a-c948d126e914 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '494cf20b9e544302a3b405ca182e5e40'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:00.291509 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1081/2162] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:51:00 2022] DELETE /identity/v3/users/494cf20b9e544302a3b405ca182e5e40 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:51:00.324643 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1082/2163] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:51:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:51:00.668685 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b95a0-9fb7-44d2-ac6a-c948d126e914 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:00.670319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-0e9b95a0-9fb7-44d2-ac6a-c948d126e914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:51:00.766790 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:51:00.785022 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b95a0-9fb7-44d2-ac6a-c948d126e914 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:00.790786 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c60b4b-1acc-4455-b3da-520998bfbe2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:00.791842 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c60b4b-1acc-4455-b3da-520998bfbe2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:00.792254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c60b4b-1acc-4455-b3da-520998bfbe2c admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:00.792693 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c60b4b-1acc-4455-b3da-520998bfbe2c admin admin] PATH_INFO: `/v3/projects/345a263c03474a16a4a44b14e44e11e9` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:00.794290 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c60b4b-1acc-4455-b3da-520998bfbe2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=345a263c03474a16a4a44b14e44e11e9)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:00.796402 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c60b4b-1acc-4455-b3da-520998bfbe2c admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:00.850140 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-68c60b4b-1acc-4455-b3da-520998bfbe2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '345a263c03474a16a4a44b14e44e11e9', 'request_id': 'req-68c60b4b-1acc-4455-b3da-520998bfbe2c'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:00.856258 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-68c60b4b-1acc-4455-b3da-520998bfbe2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '345a263c03474a16a4a44b14e44e11e9', 'request_id': 'req-68c60b4b-1acc-4455-b3da-520998bfbe2c'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:00.863377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-68c60b4b-1acc-4455-b3da-520998bfbe2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '345a263c03474a16a4a44b14e44e11e9', 'request_id': 'req-68c60b4b-1acc-4455-b3da-520998bfbe2c'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:00.878881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-68c60b4b-1acc-4455-b3da-520998bfbe2c admin admin] The token cache is being invalidate because project 345a263c03474a16a4a44b14e44e11e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:51:00.879193 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-68c60b4b-1acc-4455-b3da-520998bfbe2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:00.879980 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-68c60b4b-1acc-4455-b3da-520998bfbe2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:00.881782 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1082/2164] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:51:00 2022] DELETE /identity/v3/projects/345a263c03474a16a4a44b14e44e11e9 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:51:00.887752 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af06f33-c77b-4083-a75c-7377b199b657 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:00.888327 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af06f33-c77b-4083-a75c-7377b199b657 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:00.888846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af06f33-c77b-4083-a75c-7377b199b657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:00.911418 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-1af06f33-c77b-4083-a75c-7377b199b657 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:51:00.912806 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-1af06f33-c77b-4083-a75c-7377b199b657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:51:00.981965 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1083/2165] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:51:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:51:00.985090 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93684cc2-f9e3-4d3a-9d72-0685589cd488 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:00.985475 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93684cc2-f9e3-4d3a-9d72-0685589cd488 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:00.985687 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93684cc2-f9e3-4d3a-9d72-0685589cd488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:01.005449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-93684cc2-f9e3-4d3a-9d72-0685589cd488 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:51:01.006677 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-93684cc2-f9e3-4d3a-9d72-0685589cd488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:51:01.019729 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1083/2166] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:51:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:51:01.021186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af06f33-c77b-4083-a75c-7377b199b657 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:01.034054 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af06f33-c77b-4083-a75c-7377b199b657 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:01.039314 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12aea02-e293-4451-a378-3d883908e2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:01.039989 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12aea02-e293-4451-a378-3d883908e2cc admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:01.040445 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12aea02-e293-4451-a378-3d883908e2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:01.040721 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12aea02-e293-4451-a378-3d883908e2cc admin admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:01.041455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12aea02-e293-4451-a378-3d883908e2cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:51:01.041763 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12aea02-e293-4451-a378-3d883908e2cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:01.043439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12aea02-e293-4451-a378-3d883908e2cc admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:01.057187 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1084/2167] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:51:01.062200 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93684cc2-f9e3-4d3a-9d72-0685589cd488 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:01.072783 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93684cc2-f9e3-4d3a-9d72-0685589cd488 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:01.080212 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe71d4-4690-4382-9efe-396a2d49ba1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:01.080212 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe71d4-4690-4382-9efe-396a2d49ba1d admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:01.080212 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe71d4-4690-4382-9efe-396a2d49ba1d admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:01.080212 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe71d4-4690-4382-9efe-396a2d49ba1d admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:01.081089 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe71d4-4690-4382-9efe-396a2d49ba1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:01.083331 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe71d4-4690-4382-9efe-396a2d49ba1d admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:01.125015 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1084/2168] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:51:01 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:51:01.126851 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12aea02-e293-4451-a378-3d883908e2cc admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:01.140861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12aea02-e293-4451-a378-3d883908e2cc admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:01.216128 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a796e2-d70a-4902-a45c-9a785531c465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:01.217133 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a796e2-d70a-4902-a45c-9a785531c465 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:01.217423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a796e2-d70a-4902-a45c-9a785531c465 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:01.217706 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a796e2-d70a-4902-a45c-9a785531c465 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:01.218382 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a796e2-d70a-4902-a45c-9a785531c465 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:01.220468 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a796e2-d70a-4902-a45c-9a785531c465 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:01.244902 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1085/2169] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:51:01 2022] POST /identity/v3/users => generated 287 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:51:01.249994 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe71d4-4690-4382-9efe-396a2d49ba1d admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:01.265088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe71d4-4690-4382-9efe-396a2d49ba1d admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:01.271204 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee10aa3-c6b9-4fa1-a448-383f284608e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:01.271875 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee10aa3-c6b9-4fa1-a448-383f284608e1 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:01.272163 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee10aa3-c6b9-4fa1-a448-383f284608e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:01.272444 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee10aa3-c6b9-4fa1-a448-383f284608e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:01.273216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee10aa3-c6b9-4fa1-a448-383f284608e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:01.275033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee10aa3-c6b9-4fa1-a448-383f284608e1 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:01.286957 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1085/2170] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:51:01.291513 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a796e2-d70a-4902-a45c-9a785531c465 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:01.303296 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a796e2-d70a-4902-a45c-9a785531c465 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:01.309453 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad40948-a32f-4003-9560-9300b308e789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:01.310267 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad40948-a32f-4003-9560-9300b308e789 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:01.310549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad40948-a32f-4003-9560-9300b308e789 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:01.310811 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad40948-a32f-4003-9560-9300b308e789 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:01.311655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad40948-a32f-4003-9560-9300b308e789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:01.313471 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad40948-a32f-4003-9560-9300b308e789 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:01.316084 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:51:01.332301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1086/2171] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:51:01.334163 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee10aa3-c6b9-4fa1-a448-383f284608e1 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:01.349068 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee10aa3-c6b9-4fa1-a448-383f284608e1 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:01.355473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017412da-dd0c-4340-ae75-00e58a349da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:01.356390 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017412da-dd0c-4340-ae75-00e58a349da3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:01.356676 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017412da-dd0c-4340-ae75-00e58a349da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:01.356956 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017412da-dd0c-4340-ae75-00e58a349da3 admin admin] PATH_INFO: `/v3/projects/114da792d3f54f38b215aaeea69fd454/users/c29795aae948425b87a6852b41a80c5d/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:01.378260 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017412da-dd0c-4340-ae75-00e58a349da3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=114da792d3f54f38b215aaeea69fd454, user_id=c29795aae948425b87a6852b41a80c5d, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:01.381611 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017412da-dd0c-4340-ae75-00e58a349da3 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:01.399361 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1086/2172] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:51:01 2022] PUT /identity/v3/projects/114da792d3f54f38b215aaeea69fd454/users/c29795aae948425b87a6852b41a80c5d/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:51:01.404824 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce47ce9-c644-4392-b90d-4a1b343f126d None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:01.405253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce47ce9-c644-4392-b90d-4a1b343f126d None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:01.405495 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce47ce9-c644-4392-b90d-4a1b343f126d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:01.425962 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-cce47ce9-c644-4392-b90d-4a1b343f126d None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:51:01.427810 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-cce47ce9-c644-4392-b90d-4a1b343f126d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:51:01.519766 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1087/2173] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:51:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 14:51:01.529798 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017412da-dd0c-4340-ae75-00e58a349da3 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:01.531616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-017412da-dd0c-4340-ae75-00e58a349da3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:51:01.590799 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017412da-dd0c-4340-ae75-00e58a349da3 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:01.596434 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c543a0-f383-49f2-acca-fce50575cfe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:01.597431 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c543a0-f383-49f2-acca-fce50575cfe3 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:01.597716 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c543a0-f383-49f2-acca-fce50575cfe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:01.597994 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c543a0-f383-49f2-acca-fce50575cfe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:01.609800 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c543a0-f383-49f2-acca-fce50575cfe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:01.612184 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c543a0-f383-49f2-acca-fce50575cfe3 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:01.625667 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1087/2174] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:51:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:51:02.335126 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59713925-422c-4deb-b34f-16ce1e0d2f15 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:02.335658 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59713925-422c-4deb-b34f-16ce1e0d2f15 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:02.335790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59713925-422c-4deb-b34f-16ce1e0d2f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:02.367991 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-59713925-422c-4deb-b34f-16ce1e0d2f15 None None] MFA Rules not processed for user `c29795aae948425b87a6852b41a80c5d`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:51:02.369898 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-59713925-422c-4deb-b34f-16ce1e0d2f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:51:02.450919 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1088/2175] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:51:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:51:02.463087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c543a0-f383-49f2-acca-fce50575cfe3 service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:02.464535 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-a1c543a0-f383-49f2-acca-fce50575cfe3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:51:02.498733 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:51:02.531987 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c543a0-f383-49f2-acca-fce50575cfe3 service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:02.535863 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726735df-30e4-4e4a-8c16-79fb3f0ca55b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:02.536533 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726735df-30e4-4e4a-8c16-79fb3f0ca55b service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:02.536854 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726735df-30e4-4e4a-8c16-79fb3f0ca55b service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:02.537138 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726735df-30e4-4e4a-8c16-79fb3f0ca55b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:02.546684 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726735df-30e4-4e4a-8c16-79fb3f0ca55b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:02.548262 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726735df-30e4-4e4a-8c16-79fb3f0ca55b service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:02.558306 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1088/2176] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:51:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:51:10.281714 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484fae55-543d-4673-b47b-adf0e3341512 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:10.282250 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484fae55-543d-4673-b47b-adf0e3341512 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:10.282418 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484fae55-543d-4673-b47b-adf0e3341512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:10.302715 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-484fae55-543d-4673-b47b-adf0e3341512 None None] MFA Rules not processed for user `c29795aae948425b87a6852b41a80c5d`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:51:10.304038 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-484fae55-543d-4673-b47b-adf0e3341512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:51:10.312891 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1089/2177] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:51:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:51:12.553529 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=15 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:51:20.295672 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=64 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:51:26.168523 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726735df-30e4-4e4a-8c16-79fb3f0ca55b service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:26.170285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-726735df-30e4-4e4a-8c16-79fb3f0ca55b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:51:26.187390 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726735df-30e4-4e4a-8c16-79fb3f0ca55b service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:26.193803 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d264a19c-7872-4fd4-9dc8-5b28f20b6bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:26.195225 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d264a19c-7872-4fd4-9dc8-5b28f20b6bcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:26.195225 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d264a19c-7872-4fd4-9dc8-5b28f20b6bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:26.195444 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d264a19c-7872-4fd4-9dc8-5b28f20b6bcd admin admin] PATH_INFO: `/v3/users/45c2342246b84fd4a429335e065f813d` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:26.197007 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d264a19c-7872-4fd4-9dc8-5b28f20b6bcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=45c2342246b84fd4a429335e065f813d)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:26.199887 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d264a19c-7872-4fd4-9dc8-5b28f20b6bcd admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:26.268818 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-d264a19c-7872-4fd4-9dc8-5b28f20b6bcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45c2342246b84fd4a429335e065f813d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:26.286892 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-d264a19c-7872-4fd4-9dc8-5b28f20b6bcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45c2342246b84fd4a429335e065f813d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:26.286892 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-d264a19c-7872-4fd4-9dc8-5b28f20b6bcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45c2342246b84fd4a429335e065f813d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:26.286892 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-d264a19c-7872-4fd4-9dc8-5b28f20b6bcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45c2342246b84fd4a429335e065f813d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:26.295075 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-d264a19c-7872-4fd4-9dc8-5b28f20b6bcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45c2342246b84fd4a429335e065f813d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:26.311264 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1089/2178] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:51:26 2022] DELETE /identity/v3/users/45c2342246b84fd4a429335e065f813d => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:51:26.316269 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24a21a-56e1-4bca-aeba-1d3039877e42 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:26.316756 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24a21a-56e1-4bca-aeba-1d3039877e42 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:26.316846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24a21a-56e1-4bca-aeba-1d3039877e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:26.335417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-fb24a21a-56e1-4bca-aeba-1d3039877e42 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:51:26.336639 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-fb24a21a-56e1-4bca-aeba-1d3039877e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:51:26.414729 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1090/2179] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:51:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:51:26.424963 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d264a19c-7872-4fd4-9dc8-5b28f20b6bcd admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:26.426825 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-d264a19c-7872-4fd4-9dc8-5b28f20b6bcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:51:26.496399 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d264a19c-7872-4fd4-9dc8-5b28f20b6bcd admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:26.500172 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59b1d8-c638-4f06-b174-a23bd2a0857d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:26.500863 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59b1d8-c638-4f06-b174-a23bd2a0857d service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:26.501146 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59b1d8-c638-4f06-b174-a23bd2a0857d service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:26.501422 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59b1d8-c638-4f06-b174-a23bd2a0857d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:26.511588 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59b1d8-c638-4f06-b174-a23bd2a0857d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:26.513038 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59b1d8-c638-4f06-b174-a23bd2a0857d service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:26.523188 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1090/2180] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:51:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:51:26.888252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb24a21a-56e1-4bca-aeba-1d3039877e42 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:26.890002 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-fb24a21a-56e1-4bca-aeba-1d3039877e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:51:26.910989 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb24a21a-56e1-4bca-aeba-1d3039877e42 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:26.916897 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0144b-fc7d-4473-bcdb-239fc594553c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:26.917803 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0144b-fc7d-4473-bcdb-239fc594553c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:26.918185 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0144b-fc7d-4473-bcdb-239fc594553c admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:26.918331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0144b-fc7d-4473-bcdb-239fc594553c admin admin] PATH_INFO: `/v3/projects/ed626c0caa9941fd8e76da43dc461451` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:26.919652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0144b-fc7d-4473-bcdb-239fc594553c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed626c0caa9941fd8e76da43dc461451)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:26.921610 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0144b-fc7d-4473-bcdb-239fc594553c admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:26.978056 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-59b0144b-fc7d-4473-bcdb-239fc594553c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed626c0caa9941fd8e76da43dc461451', 'request_id': 'req-59b0144b-fc7d-4473-bcdb-239fc594553c'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:26.984213 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-59b0144b-fc7d-4473-bcdb-239fc594553c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed626c0caa9941fd8e76da43dc461451', 'request_id': 'req-59b0144b-fc7d-4473-bcdb-239fc594553c'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:26.995789 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-59b0144b-fc7d-4473-bcdb-239fc594553c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed626c0caa9941fd8e76da43dc461451', 'request_id': 'req-59b0144b-fc7d-4473-bcdb-239fc594553c'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:27.013515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-59b0144b-fc7d-4473-bcdb-239fc594553c admin admin] The token cache is being invalidate because project ed626c0caa9941fd8e76da43dc461451 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:51:27.013861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-59b0144b-fc7d-4473-bcdb-239fc594553c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:27.014582 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-59b0144b-fc7d-4473-bcdb-239fc594553c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:27.016283 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1091/2181] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:51:26 2022] DELETE /identity/v3/projects/ed626c0caa9941fd8e76da43dc461451 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:51:27.021150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241e1cb-7758-499e-acf6-b9020b288448 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:27.021936 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241e1cb-7758-499e-acf6-b9020b288448 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:27.022088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241e1cb-7758-499e-acf6-b9020b288448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:27.043362 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-f241e1cb-7758-499e-acf6-b9020b288448 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:51:27.044602 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-f241e1cb-7758-499e-acf6-b9020b288448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:51:27.113386 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1091/2182] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:51:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:51:27.121371 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b0e07-d6e7-4ebb-9cad-602c7f7c1d04 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:27.121650 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b0e07-d6e7-4ebb-9cad-602c7f7c1d04 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:27.121926 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b0e07-d6e7-4ebb-9cad-602c7f7c1d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:27.143176 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-9c9b0e07-d6e7-4ebb-9cad-602c7f7c1d04 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:51:27.144701 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-9c9b0e07-d6e7-4ebb-9cad-602c7f7c1d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:51:27.161803 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1092/2183] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:51:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:51:27.166876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241e1cb-7758-499e-acf6-b9020b288448 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:27.183172 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241e1cb-7758-499e-acf6-b9020b288448 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:27.188564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a7968-031a-448d-9a30-edbe359c17c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:27.189257 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a7968-031a-448d-9a30-edbe359c17c9 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:27.189548 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a7968-031a-448d-9a30-edbe359c17c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:27.189817 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a7968-031a-448d-9a30-edbe359c17c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:27.190636 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a7968-031a-448d-9a30-edbe359c17c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:51:27.192266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a7968-031a-448d-9a30-edbe359c17c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:27.192674 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a7968-031a-448d-9a30-edbe359c17c9 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:27.206395 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1092/2184] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:51:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:51:27.210860 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b0e07-d6e7-4ebb-9cad-602c7f7c1d04 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:27.222367 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b0e07-d6e7-4ebb-9cad-602c7f7c1d04 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:27.227294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07cf268-992c-4ed5-9510-53dbf868f5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:27.228000 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07cf268-992c-4ed5-9510-53dbf868f5fd admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:27.228287 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07cf268-992c-4ed5-9510-53dbf868f5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:27.228562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07cf268-992c-4ed5-9510-53dbf868f5fd admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:27.229319 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07cf268-992c-4ed5-9510-53dbf868f5fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:27.231302 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07cf268-992c-4ed5-9510-53dbf868f5fd admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:27.264858 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1093/2185] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:51:27 2022] POST /identity/v3/projects => generated 389 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:51:27.269963 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a7968-031a-448d-9a30-edbe359c17c9 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:27.275532 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:51:27.282970 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a7968-031a-448d-9a30-edbe359c17c9 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:27.354104 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56078484-bc05-498d-bf89-638fa559f597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:27.354813 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56078484-bc05-498d-bf89-638fa559f597 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:27.355164 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56078484-bc05-498d-bf89-638fa559f597 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:27.355450 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56078484-bc05-498d-bf89-638fa559f597 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:27.356187 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56078484-bc05-498d-bf89-638fa559f597 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:27.358348 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56078484-bc05-498d-bf89-638fa559f597 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:27.386490 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1093/2186] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:51:27 2022] POST /identity/v3/users => generated 295 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:51:27.387791 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07cf268-992c-4ed5-9510-53dbf868f5fd admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:27.398986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07cf268-992c-4ed5-9510-53dbf868f5fd admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:27.404828 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f1a2c-1419-4333-89a8-87fabacffd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:27.405517 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f1a2c-1419-4333-89a8-87fabacffd2c admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:27.405805 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f1a2c-1419-4333-89a8-87fabacffd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:27.406087 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f1a2c-1419-4333-89a8-87fabacffd2c admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:27.406834 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f1a2c-1419-4333-89a8-87fabacffd2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:27.408499 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f1a2c-1419-4333-89a8-87fabacffd2c admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:27.419778 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1094/2187] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:51:27.424106 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56078484-bc05-498d-bf89-638fa559f597 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:27.435525 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56078484-bc05-498d-bf89-638fa559f597 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:27.442338 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291d63d-e56e-439b-91fc-99ab1259ddad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:27.443240 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291d63d-e56e-439b-91fc-99ab1259ddad admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:27.443690 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291d63d-e56e-439b-91fc-99ab1259ddad admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:27.444125 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291d63d-e56e-439b-91fc-99ab1259ddad admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:27.445191 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291d63d-e56e-439b-91fc-99ab1259ddad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:27.446942 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291d63d-e56e-439b-91fc-99ab1259ddad admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:27.458833 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1094/2188] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:51:27.463361 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f1a2c-1419-4333-89a8-87fabacffd2c admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:27.476406 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f1a2c-1419-4333-89a8-87fabacffd2c admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:27.481748 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd9bc9-f584-4fe3-b8df-bcf24322662f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:27.482894 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd9bc9-f584-4fe3-b8df-bcf24322662f admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:27.483266 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd9bc9-f584-4fe3-b8df-bcf24322662f admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:27.483635 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd9bc9-f584-4fe3-b8df-bcf24322662f admin admin] PATH_INFO: `/v3/projects/d661a20979714bae8711e1e0f27653fb/users/08dd3f1e931e48a3be3d019411054cc1/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:27.499678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd9bc9-f584-4fe3-b8df-bcf24322662f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d661a20979714bae8711e1e0f27653fb, user_id=08dd3f1e931e48a3be3d019411054cc1, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:27.502632 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd9bc9-f584-4fe3-b8df-bcf24322662f admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:27.518245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1095/2189] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:51:27 2022] PUT /identity/v3/projects/d661a20979714bae8711e1e0f27653fb/users/08dd3f1e931e48a3be3d019411054cc1/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:51:27.529717 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d14624-7a58-4ff2-887c-85dc68466721 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:27.530008 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d14624-7a58-4ff2-887c-85dc68466721 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:27.530288 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d14624-7a58-4ff2-887c-85dc68466721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:27.567518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-29d14624-7a58-4ff2-887c-85dc68466721 None None] MFA Rules not processed for user `08dd3f1e931e48a3be3d019411054cc1`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:51:27.568787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-29d14624-7a58-4ff2-887c-85dc68466721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:51:27.638715 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1095/2190] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:51:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:51:27.650125 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd9bc9-f584-4fe3-b8df-bcf24322662f admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:27.652655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-8fbd9bc9-f584-4fe3-b8df-bcf24322662f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:51:27.677503 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:51:27.720199 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd9bc9-f584-4fe3-b8df-bcf24322662f admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:27.723919 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd7983-47a9-4078-b92a-3aa4a48024aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:27.724593 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd7983-47a9-4078-b92a-3aa4a48024aa service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:27.724867 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd7983-47a9-4078-b92a-3aa4a48024aa service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:27.725144 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd7983-47a9-4078-b92a-3aa4a48024aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:27.734118 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cd7983-47a9-4078-b92a-3aa4a48024aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:27.735656 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cd7983-47a9-4078-b92a-3aa4a48024aa service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:27.745991 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1096/2191] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:51:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 14:51:37.634830 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=65,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:51:37.741576 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=17 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:51:43.287011 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b6468-16b7-465e-ad25-a86cc8f818ec None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:43.287671 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b6468-16b7-465e-ad25-a86cc8f818ec None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:43.287955 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b6468-16b7-465e-ad25-a86cc8f818ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:43.306871 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-230b6468-16b7-465e-ad25-a86cc8f818ec None None] MFA Rules not processed for user `c29795aae948425b87a6852b41a80c5d`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:51:43.308150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-230b6468-16b7-465e-ad25-a86cc8f818ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:51:43.367651 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1096/2192] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:51:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:51:52.436828 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd7983-47a9-4078-b92a-3aa4a48024aa service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:52.510416 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd7983-47a9-4078-b92a-3aa4a48024aa service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:52.515674 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b3ab3d-e6c0-4fd1-b211-691ce4e9f97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:52.516471 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3ab3d-e6c0-4fd1-b211-691ce4e9f97e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:52.516828 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3ab3d-e6c0-4fd1-b211-691ce4e9f97e admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:52.517170 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3ab3d-e6c0-4fd1-b211-691ce4e9f97e admin admin] PATH_INFO: `/v3/users/08dd3f1e931e48a3be3d019411054cc1` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:52.518503 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b3ab3d-e6c0-4fd1-b211-691ce4e9f97e admin admin] RBAC: Authorizing `identity:delete_user(user_id=08dd3f1e931e48a3be3d019411054cc1)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:52.520590 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b3ab3d-e6c0-4fd1-b211-691ce4e9f97e admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:52.583339 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-48b3ab3d-e6c0-4fd1-b211-691ce4e9f97e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08dd3f1e931e48a3be3d019411054cc1'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:52.600805 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-48b3ab3d-e6c0-4fd1-b211-691ce4e9f97e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08dd3f1e931e48a3be3d019411054cc1'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:52.601606 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-48b3ab3d-e6c0-4fd1-b211-691ce4e9f97e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08dd3f1e931e48a3be3d019411054cc1'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:52.602370 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-48b3ab3d-e6c0-4fd1-b211-691ce4e9f97e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08dd3f1e931e48a3be3d019411054cc1'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:52.610743 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-48b3ab3d-e6c0-4fd1-b211-691ce4e9f97e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08dd3f1e931e48a3be3d019411054cc1'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:52.629107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1097/2193] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:51:52 2022] DELETE /identity/v3/users/08dd3f1e931e48a3be3d019411054cc1 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:51:52.633501 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02178136-1015-43b5-9222-d1d38f3b3af5 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:52.634039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02178136-1015-43b5-9222-d1d38f3b3af5 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:52.634039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02178136-1015-43b5-9222-d1d38f3b3af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:52.653221 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-02178136-1015-43b5-9222-d1d38f3b3af5 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:51:52.654452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-02178136-1015-43b5-9222-d1d38f3b3af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:51:52.739229 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1097/2194] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:51:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:51:52.747304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b3ab3d-e6c0-4fd1-b211-691ce4e9f97e admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:52.748750 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-48b3ab3d-e6c0-4fd1-b211-691ce4e9f97e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:51:52.808718 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b3ab3d-e6c0-4fd1-b211-691ce4e9f97e admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:52.812747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661bea6e-cd77-4e37-b7f9-b2dc42974f3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:52.813433 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661bea6e-cd77-4e37-b7f9-b2dc42974f3f service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:52.813696 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661bea6e-cd77-4e37-b7f9-b2dc42974f3f service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:52.813974 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661bea6e-cd77-4e37-b7f9-b2dc42974f3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:52.824006 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661bea6e-cd77-4e37-b7f9-b2dc42974f3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:52.825482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661bea6e-cd77-4e37-b7f9-b2dc42974f3f service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:52.835975 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1098/2195] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:51:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:51:53.127009 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02178136-1015-43b5-9222-d1d38f3b3af5 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:53.128537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-02178136-1015-43b5-9222-d1d38f3b3af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:51:53.145046 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02178136-1015-43b5-9222-d1d38f3b3af5 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:53.150830 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede70fde-76c8-4c6f-aa50-e6c48d1d154f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:53.152138 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede70fde-76c8-4c6f-aa50-e6c48d1d154f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:53.152478 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede70fde-76c8-4c6f-aa50-e6c48d1d154f admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:53.152808 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede70fde-76c8-4c6f-aa50-e6c48d1d154f admin admin] PATH_INFO: `/v3/projects/d661a20979714bae8711e1e0f27653fb` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:53.155809 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede70fde-76c8-4c6f-aa50-e6c48d1d154f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d661a20979714bae8711e1e0f27653fb)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:53.158366 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede70fde-76c8-4c6f-aa50-e6c48d1d154f admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:53.218862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ede70fde-76c8-4c6f-aa50-e6c48d1d154f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd661a20979714bae8711e1e0f27653fb', 'request_id': 'req-ede70fde-76c8-4c6f-aa50-e6c48d1d154f'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:53.227775 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ede70fde-76c8-4c6f-aa50-e6c48d1d154f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd661a20979714bae8711e1e0f27653fb', 'request_id': 'req-ede70fde-76c8-4c6f-aa50-e6c48d1d154f'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:53.235116 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ede70fde-76c8-4c6f-aa50-e6c48d1d154f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd661a20979714bae8711e1e0f27653fb', 'request_id': 'req-ede70fde-76c8-4c6f-aa50-e6c48d1d154f'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:53.259990 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ede70fde-76c8-4c6f-aa50-e6c48d1d154f admin admin] The token cache is being invalidate because project d661a20979714bae8711e1e0f27653fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:51:53.260270 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ede70fde-76c8-4c6f-aa50-e6c48d1d154f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:53.261035 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ede70fde-76c8-4c6f-aa50-e6c48d1d154f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:51:53.264642 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1098/2196] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:51:53 2022] DELETE /identity/v3/projects/d661a20979714bae8711e1e0f27653fb => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:51:53.269890 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8a9fb-fbbe-4675-8d5a-f85130493281 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:53.270362 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8a9fb-fbbe-4675-8d5a-f85130493281 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:53.270809 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8a9fb-fbbe-4675-8d5a-f85130493281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:53.291789 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-a9c8a9fb-fbbe-4675-8d5a-f85130493281 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:51:53.293097 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-a9c8a9fb-fbbe-4675-8d5a-f85130493281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:51:53.312771 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:51:53.377130 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1099/2197] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:51:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:51:53.385964 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc394a-9f6e-4fe4-9e64-8ff91a8e8bb1 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:53.386260 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc394a-9f6e-4fe4-9e64-8ff91a8e8bb1 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:53.386539 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc394a-9f6e-4fe4-9e64-8ff91a8e8bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:53.407199 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-61dc394a-9f6e-4fe4-9e64-8ff91a8e8bb1 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:51:53.408434 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-61dc394a-9f6e-4fe4-9e64-8ff91a8e8bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:51:53.416488 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1099/2198] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:51:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:51:53.421520 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c8a9fb-fbbe-4675-8d5a-f85130493281 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:53.437548 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c8a9fb-fbbe-4675-8d5a-f85130493281 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:53.442580 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb4e83-629d-44eb-9b95-6932344c3116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:53.443316 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb4e83-629d-44eb-9b95-6932344c3116 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:53.443603 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb4e83-629d-44eb-9b95-6932344c3116 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:53.443880 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb4e83-629d-44eb-9b95-6932344c3116 admin admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:53.444624 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb4e83-629d-44eb-9b95-6932344c3116 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:51:53.444933 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb4e83-629d-44eb-9b95-6932344c3116 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:53.446604 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb4e83-629d-44eb-9b95-6932344c3116 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:53.459118 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1100/2199] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:51:53.463484 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc394a-9f6e-4fe4-9e64-8ff91a8e8bb1 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:53.472927 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc394a-9f6e-4fe4-9e64-8ff91a8e8bb1 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:53.478529 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56388eff-98c4-4447-a82d-9062cf55f4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:53.479229 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56388eff-98c4-4447-a82d-9062cf55f4a9 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:53.479510 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56388eff-98c4-4447-a82d-9062cf55f4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:53.479787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56388eff-98c4-4447-a82d-9062cf55f4a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:53.480456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56388eff-98c4-4447-a82d-9062cf55f4a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:53.482360 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56388eff-98c4-4447-a82d-9062cf55f4a9 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:53.505141 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:51:53.516105 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1100/2200] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:51:53 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 14:51:53.517361 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb4e83-629d-44eb-9b95-6932344c3116 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:53.527808 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb4e83-629d-44eb-9b95-6932344c3116 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:53.590923 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc9695a-27f8-4b99-a894-5534c283ec25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:53.591642 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9695a-27f8-4b99-a894-5534c283ec25 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:53.591924 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9695a-27f8-4b99-a894-5534c283ec25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:53.592196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9695a-27f8-4b99-a894-5534c283ec25 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:53.592889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc9695a-27f8-4b99-a894-5534c283ec25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:53.594909 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc9695a-27f8-4b99-a894-5534c283ec25 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:53.618939 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1101/2201] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:51:53 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:51:53.620448 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56388eff-98c4-4447-a82d-9062cf55f4a9 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:53.631582 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56388eff-98c4-4447-a82d-9062cf55f4a9 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:53.636349 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb27ef-4138-4160-a8b2-bd6c9362dcbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:53.637019 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb27ef-4138-4160-a8b2-bd6c9362dcbc admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:53.637305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb27ef-4138-4160-a8b2-bd6c9362dcbc admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:53.637604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb27ef-4138-4160-a8b2-bd6c9362dcbc admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:53.638374 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb27ef-4138-4160-a8b2-bd6c9362dcbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:53.640007 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb27ef-4138-4160-a8b2-bd6c9362dcbc admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:53.650969 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1101/2202] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:51:53.655380 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc9695a-27f8-4b99-a894-5534c283ec25 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:53.665606 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc9695a-27f8-4b99-a894-5534c283ec25 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:53.670233 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4acdfad-8382-49a7-9e6f-f54d6afeb728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:53.670902 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acdfad-8382-49a7-9e6f-f54d6afeb728 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:53.671234 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acdfad-8382-49a7-9e6f-f54d6afeb728 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:53.671518 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acdfad-8382-49a7-9e6f-f54d6afeb728 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:53.672328 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4acdfad-8382-49a7-9e6f-f54d6afeb728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:53.674376 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4acdfad-8382-49a7-9e6f-f54d6afeb728 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:53.684197 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1102/2203] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 14:51:53.688438 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb27ef-4138-4160-a8b2-bd6c9362dcbc admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:53.699327 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb27ef-4138-4160-a8b2-bd6c9362dcbc admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:53.704667 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aec4a7-e170-4697-8f6d-764b3f4d583a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:53.705597 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aec4a7-e170-4697-8f6d-764b3f4d583a admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:53.705883 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aec4a7-e170-4697-8f6d-764b3f4d583a admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:53.706166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aec4a7-e170-4697-8f6d-764b3f4d583a admin admin] PATH_INFO: `/v3/projects/779e3cc39cf14719ba34b3d12cac5a01/users/58016f5cae3d45eab6f9caf161e65c4c/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:53.723479 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aec4a7-e170-4697-8f6d-764b3f4d583a admin admin] RBAC: Authorizing `identity:create_grant(project_id=779e3cc39cf14719ba34b3d12cac5a01, user_id=58016f5cae3d45eab6f9caf161e65c4c, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:53.726335 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aec4a7-e170-4697-8f6d-764b3f4d583a admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:53.741900 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1102/2204] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:51:53 2022] PUT /identity/v3/projects/779e3cc39cf14719ba34b3d12cac5a01/users/58016f5cae3d45eab6f9caf161e65c4c/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 14:51:53.747377 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77bf409-5e5a-4085-a2b0-a406236e92b1 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:53.747670 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77bf409-5e5a-4085-a2b0-a406236e92b1 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:53.747982 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77bf409-5e5a-4085-a2b0-a406236e92b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:53.768917 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-f77bf409-5e5a-4085-a2b0-a406236e92b1 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:51:53.770974 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-f77bf409-5e5a-4085-a2b0-a406236e92b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:51:53.844241 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1103/2205] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:51:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:51:53.855910 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aec4a7-e170-4697-8f6d-764b3f4d583a admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:53.857666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-56aec4a7-e170-4697-8f6d-764b3f4d583a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:51:53.922489 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aec4a7-e170-4697-8f6d-764b3f4d583a admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:53.926160 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416256eb-9c2b-43f2-9402-11bf7f9b9980 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:53.926831 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416256eb-9c2b-43f2-9402-11bf7f9b9980 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:53.927149 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416256eb-9c2b-43f2-9402-11bf7f9b9980 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:53.927430 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416256eb-9c2b-43f2-9402-11bf7f9b9980 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:53.942224 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416256eb-9c2b-43f2-9402-11bf7f9b9980 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:53.943818 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416256eb-9c2b-43f2-9402-11bf7f9b9980 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:53.954588 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1103/2206] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:51:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:51:58.560456 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737902a-84f7-47cb-a5fe-9357d42ad568 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:58.560999 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737902a-84f7-47cb-a5fe-9357d42ad568 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:58.561148 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737902a-84f7-47cb-a5fe-9357d42ad568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:58.602495 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-3737902a-84f7-47cb-a5fe-9357d42ad568 None None] MFA Rules not processed for user `58016f5cae3d45eab6f9caf161e65c4c`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:51:58.603922 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-3737902a-84f7-47cb-a5fe-9357d42ad568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:51:58.626673 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:51:58.688456 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1104/2207] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:51:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:51:58.693228 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416256eb-9c2b-43f2-9402-11bf7f9b9980 service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:58.704618 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416256eb-9c2b-43f2-9402-11bf7f9b9980 service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:58.710090 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6b50f3-4669-4492-802d-057f22bc32a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:58.710854 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b50f3-4669-4492-802d-057f22bc32a7 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:58.711189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b50f3-4669-4492-802d-057f22bc32a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:58.711486 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b50f3-4669-4492-802d-057f22bc32a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:58.712182 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6b50f3-4669-4492-802d-057f22bc32a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:58.714350 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6b50f3-4669-4492-802d-057f22bc32a7 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:58.749865 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1104/2208] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:51:58 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:51:58.755711 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737902a-84f7-47cb-a5fe-9357d42ad568 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:58.769440 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737902a-84f7-47cb-a5fe-9357d42ad568 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:58.835854 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7069c-0c69-4a75-90ff-93cae5fc1269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:58.836665 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7069c-0c69-4a75-90ff-93cae5fc1269 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:58.837126 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7069c-0c69-4a75-90ff-93cae5fc1269 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:58.837467 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7069c-0c69-4a75-90ff-93cae5fc1269 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:58.838267 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b7069c-0c69-4a75-90ff-93cae5fc1269 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:58.840341 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b7069c-0c69-4a75-90ff-93cae5fc1269 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:58.863454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1105/2209] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:51:58 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 14:51:58.876067 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6b50f3-4669-4492-802d-057f22bc32a7 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:58.889299 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6b50f3-4669-4492-802d-057f22bc32a7 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:58.896549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6a2e2c-908a-4d0a-bcb5-ecfdb2f97ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:58.897278 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a2e2c-908a-4d0a-bcb5-ecfdb2f97ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:58.897567 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a2e2c-908a-4d0a-bcb5-ecfdb2f97ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:58.897838 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a2e2c-908a-4d0a-bcb5-ecfdb2f97ee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:58.898614 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6a2e2c-908a-4d0a-bcb5-ecfdb2f97ee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:58.900350 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6a2e2c-908a-4d0a-bcb5-ecfdb2f97ee3 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:58.913942 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1105/2210] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:51:58.918262 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7069c-0c69-4a75-90ff-93cae5fc1269 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:58.929254 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7069c-0c69-4a75-90ff-93cae5fc1269 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:58.934289 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1320b3-ba46-45d6-adde-23a4ece92d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:58.935391 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1320b3-ba46-45d6-adde-23a4ece92d3d admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:58.935783 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1320b3-ba46-45d6-adde-23a4ece92d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:58.936139 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1320b3-ba46-45d6-adde-23a4ece92d3d admin admin] PATH_INFO: `/v3/projects/e74f19cd54e84ccba8aa971edec4fb5e/users/18e5f9018c0143c991414b649a4c1260/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:58.953350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1320b3-ba46-45d6-adde-23a4ece92d3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e74f19cd54e84ccba8aa971edec4fb5e, user_id=18e5f9018c0143c991414b649a4c1260, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:58.956406 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1320b3-ba46-45d6-adde-23a4ece92d3d admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:58.973461 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1106/2211] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:51:58 2022] PUT /identity/v3/projects/e74f19cd54e84ccba8aa971edec4fb5e/users/18e5f9018c0143c991414b649a4c1260/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 14:51:58.977740 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6a2e2c-908a-4d0a-bcb5-ecfdb2f97ee3 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:58.989054 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6a2e2c-908a-4d0a-bcb5-ecfdb2f97ee3 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:59.050512 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34fefc-c23c-4a90-9f23-117d1ab12c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:59.051272 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34fefc-c23c-4a90-9f23-117d1ab12c8f admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:59.051557 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34fefc-c23c-4a90-9f23-117d1ab12c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:59.051838 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34fefc-c23c-4a90-9f23-117d1ab12c8f admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:59.052587 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34fefc-c23c-4a90-9f23-117d1ab12c8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:59.054426 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34fefc-c23c-4a90-9f23-117d1ab12c8f admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:59.064897 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1106/2212] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:51:59.069389 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1320b3-ba46-45d6-adde-23a4ece92d3d admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:51:59.084435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1320b3-ba46-45d6-adde-23a4ece92d3d admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:51:59.091976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c38f9c-e877-4d91-93ab-a6e03b721936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:51:59.092938 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c38f9c-e877-4d91-93ab-a6e03b721936 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:51:59.093227 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c38f9c-e877-4d91-93ab-a6e03b721936 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:51:59.093621 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c38f9c-e877-4d91-93ab-a6e03b721936 admin admin] PATH_INFO: `/v3/projects/e74f19cd54e84ccba8aa971edec4fb5e/users/18e5f9018c0143c991414b649a4c1260/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:51:59.095810 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c38f9c-e877-4d91-93ab-a6e03b721936 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e74f19cd54e84ccba8aa971edec4fb5e, user_id=18e5f9018c0143c991414b649a4c1260, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:51:59.098963 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c38f9c-e877-4d91-93ab-a6e03b721936 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:51:59.116924 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1107/2213] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:51:59 2022] PUT /identity/v3/projects/e74f19cd54e84ccba8aa971edec4fb5e/users/18e5f9018c0143c991414b649a4c1260/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:52:04.425437 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897ca3f-f7d0-42a5-a4d7-bc5ef15156eb None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:52:04.425990 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897ca3f-f7d0-42a5-a4d7-bc5ef15156eb None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:52:04.426123 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897ca3f-f7d0-42a5-a4d7-bc5ef15156eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:52:04.453184 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-5897ca3f-f7d0-42a5-a4d7-bc5ef15156eb None None] MFA Rules not processed for user `18e5f9018c0143c991414b649a4c1260`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:52:04.454432 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-5897ca3f-f7d0-42a5-a4d7-bc5ef15156eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:52:04.474201 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:52:04.546477 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1107/2214] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:52:04 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:52:04.560261 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c38f9c-e877-4d91-93ab-a6e03b721936 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:52:04.562406 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-a8c38f9c-e877-4d91-93ab-a6e03b721936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:52:04.672566 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c38f9c-e877-4d91-93ab-a6e03b721936 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:52:04.679029 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf523f-4059-45fd-84a2-34927c3eed9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:52:04.679755 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf523f-4059-45fd-84a2-34927c3eed9b service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:52:04.680044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf523f-4059-45fd-84a2-34927c3eed9b service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:52:04.680326 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf523f-4059-45fd-84a2-34927c3eed9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:52:04.686075 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:52:04.744410 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf523f-4059-45fd-84a2-34927c3eed9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:52:04.746045 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf523f-4059-45fd-84a2-34927c3eed9b service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:52:04.756057 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1108/2215] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:52:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:52:05.928402 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26436167-0476-4bac-bb41-0f5c5a2f8ff4 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:52:05.928955 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26436167-0476-4bac-bb41-0f5c5a2f8ff4 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:52:05.928955 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26436167-0476-4bac-bb41-0f5c5a2f8ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:52:05.947960 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-26436167-0476-4bac-bb41-0f5c5a2f8ff4 None None] MFA Rules not processed for user `58016f5cae3d45eab6f9caf161e65c4c`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:52:05.949154 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-26436167-0476-4bac-bb41-0f5c5a2f8ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:52:05.956736 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1108/2216] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:52:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:52:10.270760 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527a3ab-fc0c-49e9-ab55-902fcbaa4b79 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:52:10.271542 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527a3ab-fc0c-49e9-ab55-902fcbaa4b79 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:52:10.271718 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527a3ab-fc0c-49e9-ab55-902fcbaa4b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:52:10.292396 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-c527a3ab-fc0c-49e9-ab55-902fcbaa4b79 None None] MFA Rules not processed for user `2f8cf6d908314281a3ddc9d9513d39c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:52:10.293634 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-c527a3ab-fc0c-49e9-ab55-902fcbaa4b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:52:10.356538 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1109/2217] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Jul 20 14:52:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:52:10.411359 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26436167-0476-4bac-bb41-0f5c5a2f8ff4 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:52:10.413135 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-26436167-0476-4bac-bb41-0f5c5a2f8ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:52:10.504257 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26436167-0476-4bac-bb41-0f5c5a2f8ff4 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:52:10.511438 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1155d2de-6e8c-4956-b796-60fbd49c5a9c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f4844988474d7a91133d77906611a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:52:10.512151 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1155d2de-6e8c-4956-b796-60fbd49c5a9c service swift] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:52:10.512453 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1155d2de-6e8c-4956-b796-60fbd49c5a9c service swift] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:52:10.512720 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1155d2de-6e8c-4956-b796-60fbd49c5a9c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:52:10.523485 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1155d2de-6e8c-4956-b796-60fbd49c5a9c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:52:10.525440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1155d2de-6e8c-4956-b796-60fbd49c5a9c service swift] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:52:10.540474 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1109/2218] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jul 20 14:52:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 14:52:20.348939 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=49 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:52:20.538250 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=55 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:52:33.897741 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f5bccd-cf56-41fc-a16e-38fe7453767a None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:52:33.898291 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f5bccd-cf56-41fc-a16e-38fe7453767a None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:52:33.898291 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f5bccd-cf56-41fc-a16e-38fe7453767a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:52:33.917729 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-09f5bccd-cf56-41fc-a16e-38fe7453767a None None] MFA Rules not processed for user `2f8cf6d908314281a3ddc9d9513d39c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:52:33.918971 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-09f5bccd-cf56-41fc-a16e-38fe7453767a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:52:33.927904 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1110/2219] 10.222.0.15 () {58 vars in 967 bytes} [Wed Jul 20 14:52:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:52:33.941841 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1155d2de-6e8c-4956-b796-60fbd49c5a9c service swift] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:52:33.953384 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1155d2de-6e8c-4956-b796-60fbd49c5a9c service swift] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:52:33.962970 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0aa86-ee06-4e1c-a133-348b24053123 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f4844988474d7a91133d77906611a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:52:33.963834 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0aa86-ee06-4e1c-a133-348b24053123 service swift] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:52:33.963834 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0aa86-ee06-4e1c-a133-348b24053123 service swift] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:52:33.964053 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0aa86-ee06-4e1c-a133-348b24053123 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:52:33.978326 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e0aa86-ee06-4e1c-a133-348b24053123 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:52:33.980485 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e0aa86-ee06-4e1c-a133-348b24053123 service swift] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:52:33.991579 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1110/2220] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jul 20 14:52:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 14:52:34.627820 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35ea27-1a3c-48b2-8abc-34d189578022 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:52:34.628113 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35ea27-1a3c-48b2-8abc-34d189578022 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:52:34.628397 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35ea27-1a3c-48b2-8abc-34d189578022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:52:34.647650 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-3f35ea27-1a3c-48b2-8abc-34d189578022 None None] MFA Rules not processed for user `e3574b5b8f3241b192b0b810122d4769`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:52:34.648880 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-3f35ea27-1a3c-48b2-8abc-34d189578022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:52:34.710293 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1111/2221] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:52:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:52:36.081567 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b086f-d8b9-4f2d-80bf-3dab0b289732 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:52:36.082427 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b086f-d8b9-4f2d-80bf-3dab0b289732 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:52:36.082652 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b086f-d8b9-4f2d-80bf-3dab0b289732 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:52:36.084901 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1111/2222] 10.222.0.15 () {54 vars in 921 bytes} [Wed Jul 20 14:52:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:52:36.130789 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2729e5-4e84-4db5-ae99-b7a5835dffc7 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:52:36.131382 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2729e5-4e84-4db5-ae99-b7a5835dffc7 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:52:36.131382 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2729e5-4e84-4db5-ae99-b7a5835dffc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:52:36.157232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-6a2729e5-4e84-4db5-ae99-b7a5835dffc7 None None] MFA Rules not processed for user `cb857f32bbc04d82bfa681bd5659765f`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:52:36.158485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-6a2729e5-4e84-4db5-ae99-b7a5835dffc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:52:36.178401 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b086f-d8b9-4f2d-80bf-3dab0b289732 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:52:36.181998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-9e1b086f-d8b9-4f2d-80bf-3dab0b289732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:52:36.253097 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1112/2223] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Jul 20 14:52:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:52:36.256599 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b086f-d8b9-4f2d-80bf-3dab0b289732 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:52:36.265063 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29904b0b-50b9-40fc-8347-d59b049ba39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:52:36.265457 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2729e5-4e84-4db5-ae99-b7a5835dffc7 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:52:36.266095 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29904b0b-50b9-40fc-8347-d59b049ba39a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:52:36.266382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29904b0b-50b9-40fc-8347-d59b049ba39a admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:52:36.266987 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29904b0b-50b9-40fc-8347-d59b049ba39a admin admin] PATH_INFO: `/v3/users/c29795aae948425b87a6852b41a80c5d` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:52:36.267828 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-6a2729e5-4e84-4db5-ae99-b7a5835dffc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:52:36.268367 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29904b0b-50b9-40fc-8347-d59b049ba39a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c29795aae948425b87a6852b41a80c5d)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:52:36.270298 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29904b0b-50b9-40fc-8347-d59b049ba39a admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:52:36.283354 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2729e5-4e84-4db5-ae99-b7a5835dffc7 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:52:36.293792 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af81f5-1a42-443d-8cd1-f0de8f8f9f2c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb857f32bbc04d82bfa681bd5659765f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:52:36.294472 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af81f5-1a42-443d-8cd1-f0de8f8f9f2c service placement] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:52:36.294766 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af81f5-1a42-443d-8cd1-f0de8f8f9f2c service placement] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:52:36.295036 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af81f5-1a42-443d-8cd1-f0de8f8f9f2c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:52:36.306817 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37af81f5-1a42-443d-8cd1-f0de8f8f9f2c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:52:36.308956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37af81f5-1a42-443d-8cd1-f0de8f8f9f2c service placement] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:52:36.325006 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1113/2224] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:52:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:52:36.331370 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-29904b0b-50b9-40fc-8347-d59b049ba39a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c29795aae948425b87a6852b41a80c5d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:52:36.348978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-29904b0b-50b9-40fc-8347-d59b049ba39a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c29795aae948425b87a6852b41a80c5d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:52:36.349725 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-29904b0b-50b9-40fc-8347-d59b049ba39a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c29795aae948425b87a6852b41a80c5d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:52:36.350402 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-29904b0b-50b9-40fc-8347-d59b049ba39a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c29795aae948425b87a6852b41a80c5d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:52:36.358582 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-29904b0b-50b9-40fc-8347-d59b049ba39a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c29795aae948425b87a6852b41a80c5d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:52:36.374848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1112/2225] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 14:52:36 2022] DELETE /identity/v3/users/c29795aae948425b87a6852b41a80c5d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 14:52:36.699923 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af81f5-1a42-443d-8cd1-f0de8f8f9f2c service placement] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:52:36.724175 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af81f5-1a42-443d-8cd1-f0de8f8f9f2c service placement] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:52:36.795368 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6266a77d-3052-44e9-8ca1-2545ace56639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:52:36.796305 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6266a77d-3052-44e9-8ca1-2545ace56639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:52:36.796562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6266a77d-3052-44e9-8ca1-2545ace56639 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:52:36.796841 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6266a77d-3052-44e9-8ca1-2545ace56639 admin admin] PATH_INFO: `/v3/projects/114da792d3f54f38b215aaeea69fd454` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:52:36.798309 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6266a77d-3052-44e9-8ca1-2545ace56639 admin admin] RBAC: Authorizing `identity:delete_project(project_id=114da792d3f54f38b215aaeea69fd454)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:52:36.800319 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6266a77d-3052-44e9-8ca1-2545ace56639 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:52:36.826140 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:52:36.849734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-6266a77d-3052-44e9-8ca1-2545ace56639 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '114da792d3f54f38b215aaeea69fd454', 'request_id': 'req-6266a77d-3052-44e9-8ca1-2545ace56639'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:52:36.856434 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-6266a77d-3052-44e9-8ca1-2545ace56639 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '114da792d3f54f38b215aaeea69fd454', 'request_id': 'req-6266a77d-3052-44e9-8ca1-2545ace56639'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:52:36.865693 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-6266a77d-3052-44e9-8ca1-2545ace56639 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '114da792d3f54f38b215aaeea69fd454', 'request_id': 'req-6266a77d-3052-44e9-8ca1-2545ace56639'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:52:36.878534 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-6266a77d-3052-44e9-8ca1-2545ace56639 admin admin] The token cache is being invalidate because project 114da792d3f54f38b215aaeea69fd454 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 14:52:36.878829 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-6266a77d-3052-44e9-8ca1-2545ace56639 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:52:36.879558 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-6266a77d-3052-44e9-8ca1-2545ace56639 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 14:52:36.881154 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1114/2226] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 14:52:36 2022] DELETE /identity/v3/projects/114da792d3f54f38b215aaeea69fd454 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 14:52:36.887510 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9fd10-d0f4-4707-b0e1-8bdae3a61418 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:52:36.887835 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9fd10-d0f4-4707-b0e1-8bdae3a61418 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:52:36.888146 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9fd10-d0f4-4707-b0e1-8bdae3a61418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:52:36.908452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-14e9fd10-d0f4-4707-b0e1-8bdae3a61418 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:52:36.909691 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-14e9fd10-d0f4-4707-b0e1-8bdae3a61418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:52:36.980688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1113/2227] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:52:36.989662 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509cc1dd-9302-428e-bb06-f129d1dcfb6a None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:52:36.989953 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509cc1dd-9302-428e-bb06-f129d1dcfb6a None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:52:36.990298 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509cc1dd-9302-428e-bb06-f129d1dcfb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:52:37.009433 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-509cc1dd-9302-428e-bb06-f129d1dcfb6a None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:52:37.010643 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-509cc1dd-9302-428e-bb06-f129d1dcfb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:52:37.019828 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1115/2228] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:52:37.024566 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9fd10-d0f4-4707-b0e1-8bdae3a61418 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:52:37.035584 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9fd10-d0f4-4707-b0e1-8bdae3a61418 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:52:37.043087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e5759-75db-499c-8b1c-566da9080fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:52:37.043884 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e5759-75db-499c-8b1c-566da9080fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:52:37.044237 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e5759-75db-499c-8b1c-566da9080fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:52:37.044585 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e5759-75db-499c-8b1c-566da9080fb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:52:37.045518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88e5759-75db-499c-8b1c-566da9080fb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 14:52:37.045901 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88e5759-75db-499c-8b1c-566da9080fb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:52:37.047642 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88e5759-75db-499c-8b1c-566da9080fb9 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:52:37.049435 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:52:37.065189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1114/2229] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 14:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 14:52:37.069436 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509cc1dd-9302-428e-bb06-f129d1dcfb6a None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:52:37.079390 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509cc1dd-9302-428e-bb06-f129d1dcfb6a None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:52:37.084358 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115359fa-67e4-4640-9120-7b1a207076e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:52:37.085123 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115359fa-67e4-4640-9120-7b1a207076e2 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:52:37.085430 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115359fa-67e4-4640-9120-7b1a207076e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:52:37.085709 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115359fa-67e4-4640-9120-7b1a207076e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:52:37.086415 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115359fa-67e4-4640-9120-7b1a207076e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:52:37.088400 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115359fa-67e4-4640-9120-7b1a207076e2 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:52:37.121429 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1116/2230] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 14:52:37 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 14:52:37.127987 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e5759-75db-499c-8b1c-566da9080fb9 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:52:37.138623 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e5759-75db-499c-8b1c-566da9080fb9 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:52:37.229617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b68319-7575-4298-9dd2-ae28a412851e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:52:37.229617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b68319-7575-4298-9dd2-ae28a412851e admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:52:37.229617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b68319-7575-4298-9dd2-ae28a412851e admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:52:37.229617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b68319-7575-4298-9dd2-ae28a412851e admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:52:37.230092 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b68319-7575-4298-9dd2-ae28a412851e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:52:37.230797 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b68319-7575-4298-9dd2-ae28a412851e admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:52:37.252107 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1115/2231] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 14:52:37 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 14:52:37.256420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115359fa-67e4-4640-9120-7b1a207076e2 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:52:37.279825 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115359fa-67e4-4640-9120-7b1a207076e2 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:52:37.285757 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd8fb3-c86b-440e-898f-cd5d4c399aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:52:37.286453 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd8fb3-c86b-440e-898f-cd5d4c399aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:52:37.286738 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd8fb3-c86b-440e-898f-cd5d4c399aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:52:37.287017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd8fb3-c86b-440e-898f-cd5d4c399aa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:52:37.287879 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd8fb3-c86b-440e-898f-cd5d4c399aa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:52:37.289513 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd8fb3-c86b-440e-898f-cd5d4c399aa1 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:52:37.305738 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1117/2232] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 14:52:37.309568 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b68319-7575-4298-9dd2-ae28a412851e admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:52:37.322416 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b68319-7575-4298-9dd2-ae28a412851e admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:52:37.331292 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5753c729-8865-460c-84cb-7280ca62c623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:52:37.331912 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753c729-8865-460c-84cb-7280ca62c623 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:52:37.332210 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753c729-8865-460c-84cb-7280ca62c623 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:52:37.332472 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753c729-8865-460c-84cb-7280ca62c623 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:52:37.333230 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5753c729-8865-460c-84cb-7280ca62c623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:52:37.334969 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5753c729-8865-460c-84cb-7280ca62c623 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:52:37.346861 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1116/2233] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 14:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 14:52:37.351322 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd8fb3-c86b-440e-898f-cd5d4c399aa1 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:52:37.362231 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd8fb3-c86b-440e-898f-cd5d4c399aa1 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:52:37.367155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f5009-3206-4063-9284-a76732c7015e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:52:37.368124 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f5009-3206-4063-9284-a76732c7015e admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:52:37.368470 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f5009-3206-4063-9284-a76732c7015e admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:52:37.368808 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f5009-3206-4063-9284-a76732c7015e admin admin] PATH_INFO: `/v3/projects/1cdab962e8b0462b8d1d28dcde703c15/users/bc2f1a0d431341d6a5d1a8c5de131ece/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:52:37.385734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f5009-3206-4063-9284-a76732c7015e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cdab962e8b0462b8d1d28dcde703c15, user_id=bc2f1a0d431341d6a5d1a8c5de131ece, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:52:37.388538 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f5009-3206-4063-9284-a76732c7015e admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:52:37.410025 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1118/2234] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 14:52:37 2022] PUT /identity/v3/projects/1cdab962e8b0462b8d1d28dcde703c15/users/bc2f1a0d431341d6a5d1a8c5de131ece/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 14:52:37.416239 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66078440-014c-40b4-a195-0c42f5d275f9 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:52:37.416561 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66078440-014c-40b4-a195-0c42f5d275f9 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:52:37.417053 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66078440-014c-40b4-a195-0c42f5d275f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:52:37.440252 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-66078440-014c-40b4-a195-0c42f5d275f9 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:52:37.441537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-66078440-014c-40b4-a195-0c42f5d275f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:52:37.534034 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1117/2235] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:52:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:52:37.544767 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f5009-3206-4063-9284-a76732c7015e admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:52:37.548728 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-973f5009-3206-4063-9284-a76732c7015e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:52:37.628077 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f5009-3206-4063-9284-a76732c7015e admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:52:37.633961 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70becdb1-770b-4d02-87ad-74e2b05c3d57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:52:37.634852 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70becdb1-770b-4d02-87ad-74e2b05c3d57 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:52:37.637786 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70becdb1-770b-4d02-87ad-74e2b05c3d57 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:52:37.637786 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70becdb1-770b-4d02-87ad-74e2b05c3d57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:52:37.649279 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70becdb1-770b-4d02-87ad-74e2b05c3d57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:52:37.651390 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70becdb1-770b-4d02-87ad-74e2b05c3d57 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:52:37.667560 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1119/2236] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:52:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:52:38.665561 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90f94d-8d39-473d-ada8-6b493f69615c None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:52:38.666124 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90f94d-8d39-473d-ada8-6b493f69615c None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:52:38.666124 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90f94d-8d39-473d-ada8-6b493f69615c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:52:38.698209 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-3c90f94d-8d39-473d-ada8-6b493f69615c None None] MFA Rules not processed for user `bc2f1a0d431341d6a5d1a8c5de131ece`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:52:38.699706 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-3c90f94d-8d39-473d-ada8-6b493f69615c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:52:38.773667 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1118/2237] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:52:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:52:38.784956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70becdb1-770b-4d02-87ad-74e2b05c3d57 service neutron] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:52:38.786526 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-70becdb1-770b-4d02-87ad-74e2b05c3d57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:52:38.866370 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70becdb1-770b-4d02-87ad-74e2b05c3d57 service neutron] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:52:38.872315 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8665c-14d4-4a37-bf37-e026168b2bf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:52:38.874475 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8665c-14d4-4a37-bf37-e026168b2bf6 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:52:38.879191 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8665c-14d4-4a37-bf37-e026168b2bf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:52:38.879967 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8665c-14d4-4a37-bf37-e026168b2bf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:52:38.888506 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:52:38.894733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e8665c-14d4-4a37-bf37-e026168b2bf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:52:38.903434 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e8665c-14d4-4a37-bf37-e026168b2bf6 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:52:38.918990 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1120/2238] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:52:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:52:40.309916 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5e99f-93cb-45c8-9dbd-0ecdc01dbbc3 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:52:40.310502 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5e99f-93cb-45c8-9dbd-0ecdc01dbbc3 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:52:40.310602 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5e99f-93cb-45c8-9dbd-0ecdc01dbbc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:52:40.316865 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:52:40.332140 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-72f5e99f-93cb-45c8-9dbd-0ecdc01dbbc3 None None] MFA Rules not processed for user `bc2f1a0d431341d6a5d1a8c5de131ece`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:52:40.333402 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-72f5e99f-93cb-45c8-9dbd-0ecdc01dbbc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:52:40.340546 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1119/2239] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:52:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:52:48.095170 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8665c-14d4-4a37-bf37-e026168b2bf6 service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:52:48.105368 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8665c-14d4-4a37-bf37-e026168b2bf6 service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:52:48.109511 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d43a36-09b3-4411-9779-445a1aaff1af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:52:48.110083 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d43a36-09b3-4411-9779-445a1aaff1af service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:52:48.110371 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d43a36-09b3-4411-9779-445a1aaff1af service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:52:48.110652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d43a36-09b3-4411-9779-445a1aaff1af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:52:48.112661 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-a2d43a36-09b3-4411-9779-445a1aaff1af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:52:48.197529 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d43a36-09b3-4411-9779-445a1aaff1af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:52:48.200182 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d43a36-09b3-4411-9779-445a1aaff1af service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:52:48.214803 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1121/2240] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:52:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:52:48.381085 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f5e99f-93cb-45c8-9dbd-0ecdc01dbbc3 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:52:48.391528 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f5e99f-93cb-45c8-9dbd-0ecdc01dbbc3 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:52:48.395353 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed46dc-be13-41ea-97b4-c63c25195ee1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:52:48.396124 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed46dc-be13-41ea-97b4-c63c25195ee1 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:52:48.396513 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed46dc-be13-41ea-97b4-c63c25195ee1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:52:48.396893 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed46dc-be13-41ea-97b4-c63c25195ee1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:52:48.398923 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e1ed46dc-be13-41ea-97b4-c63c25195ee1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:52:48.416007 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ed46dc-be13-41ea-97b4-c63c25195ee1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:52:48.417753 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ed46dc-be13-41ea-97b4-c63c25195ee1 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:52:48.431950 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1120/2241] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:52:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:52:55.235948 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69ab6c-c8ff-48a5-ada2-645906af6f98 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:52:55.236462 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69ab6c-c8ff-48a5-ada2-645906af6f98 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:52:55.236577 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69ab6c-c8ff-48a5-ada2-645906af6f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:52:55.259512 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-7d69ab6c-c8ff-48a5-ada2-645906af6f98 None None] MFA Rules not processed for user `2f8cf6d908314281a3ddc9d9513d39c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:52:55.260751 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-7d69ab6c-c8ff-48a5-ada2-645906af6f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:52:55.325875 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1122/2242] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Jul 20 14:52:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:52:55.338641 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed46dc-be13-41ea-97b4-c63c25195ee1 service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:52:55.340363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e1ed46dc-be13-41ea-97b4-c63c25195ee1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:52:55.411429 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed46dc-be13-41ea-97b4-c63c25195ee1 service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:52:55.416936 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ed6038-464c-47a0-b435-80542d82e1ff service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f4844988474d7a91133d77906611a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:52:55.417607 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed6038-464c-47a0-b435-80542d82e1ff service swift] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:52:55.417894 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed6038-464c-47a0-b435-80542d82e1ff service swift] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:52:55.418181 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed6038-464c-47a0-b435-80542d82e1ff service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:52:55.427699 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ed6038-464c-47a0-b435-80542d82e1ff service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:52:55.429239 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ed6038-464c-47a0-b435-80542d82e1ff service swift] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:52:55.437738 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1121/2243] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jul 20 14:52:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 14:53:05.238682 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d69ab6c-c8ff-48a5-ada2-645906af6f98 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:53:05.240171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-7d69ab6c-c8ff-48a5-ada2-645906af6f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:53:05.266904 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d69ab6c-c8ff-48a5-ada2-645906af6f98 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:53:05.272549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b36f94b-deeb-45b1-b800-8bf40e9bf246 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:53:05.273117 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36f94b-deeb-45b1-b800-8bf40e9bf246 service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:53:05.273398 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36f94b-deeb-45b1-b800-8bf40e9bf246 service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:53:05.273670 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36f94b-deeb-45b1-b800-8bf40e9bf246 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:53:05.275855 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-9b36f94b-deeb-45b1-b800-8bf40e9bf246 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:53:05.371259 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b36f94b-deeb-45b1-b800-8bf40e9bf246 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:53:05.373463 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b36f94b-deeb-45b1-b800-8bf40e9bf246 service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:53:05.385442 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1123/2244] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:53:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:53:05.434487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=40 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:53:12.365036 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ed6038-464c-47a0-b435-80542d82e1ff service swift] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:53:12.375971 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ed6038-464c-47a0-b435-80542d82e1ff service swift] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:53:12.380242 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9da590-2d94-48e8-951a-d8ff1daf4633 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:53:12.380940 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9da590-2d94-48e8-951a-d8ff1daf4633 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:53:12.381227 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9da590-2d94-48e8-951a-d8ff1daf4633 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:53:12.381503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9da590-2d94-48e8-951a-d8ff1daf4633 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:53:12.385684 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-0a9da590-2d94-48e8-951a-d8ff1daf4633 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:53:12.394062 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9da590-2d94-48e8-951a-d8ff1daf4633 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:53:12.395620 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9da590-2d94-48e8-951a-d8ff1daf4633 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:53:12.405192 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1122/2245] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:53:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 14:53:12.546134 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbc86f-ef9a-47dc-a04a-4de70182c854 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:53:12.547272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbc86f-ef9a-47dc-a04a-4de70182c854 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:53:12.547708 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbc86f-ef9a-47dc-a04a-4de70182c854 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:53:12.549454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1124/2246] 10.222.0.15 () {54 vars in 921 bytes} [Wed Jul 20 14:53:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:53:12.558675 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383c91b-4095-4a26-a66d-d28bd513ce0a None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:53:12.558899 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383c91b-4095-4a26-a66d-d28bd513ce0a None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:53:12.559263 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383c91b-4095-4a26-a66d-d28bd513ce0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:53:12.580469 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-0383c91b-4095-4a26-a66d-d28bd513ce0a None None] MFA Rules not processed for user `5d4a107ed04348b59d20d1703dbe04eb`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:53:12.582033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-0383c91b-4095-4a26-a66d-d28bd513ce0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:53:12.593825 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1123/2247] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Jul 20 14:53:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:53:12.610188 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fbc86f-ef9a-47dc-a04a-4de70182c854 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:53:12.612009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-89fbc86f-ef9a-47dc-a04a-4de70182c854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:53:12.619554 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=91 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:53:12.688499 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fbc86f-ef9a-47dc-a04a-4de70182c854 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:53:12.692229 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62210b-4b19-494f-9e75-a2e6c2fe207f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d7d59080cd4832a45dbb6c0349cf4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:53:12.692953 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62210b-4b19-494f-9e75-a2e6c2fe207f service glance] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:53:12.693241 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62210b-4b19-494f-9e75-a2e6c2fe207f service glance] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:53:12.693517 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62210b-4b19-494f-9e75-a2e6c2fe207f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:53:12.705115 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d62210b-4b19-494f-9e75-a2e6c2fe207f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:53:12.706595 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d62210b-4b19-494f-9e75-a2e6c2fe207f service glance] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:53:12.718598 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1125/2248] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:53:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:53:12.808399 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef273c1-35bc-409f-888f-0d4020d5c546 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:53:12.808668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef273c1-35bc-409f-888f-0d4020d5c546 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:53:12.809023 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef273c1-35bc-409f-888f-0d4020d5c546 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:53:12.810371 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1124/2249] 10.222.0.15 () {54 vars in 921 bytes} [Wed Jul 20 14:53:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:53:12.815769 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da508225-de95-4d5d-903c-537561be0554 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:53:12.816060 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da508225-de95-4d5d-903c-537561be0554 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:53:12.816336 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da508225-de95-4d5d-903c-537561be0554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:53:12.840120 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-da508225-de95-4d5d-903c-537561be0554 None None] MFA Rules not processed for user `7658ffde740d446682c1876662815987`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:53:12.841438 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-da508225-de95-4d5d-903c-537561be0554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:53:12.849309 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1126/2250] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Jul 20 14:53:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:53:12.861725 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef273c1-35bc-409f-888f-0d4020d5c546 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:53:12.874072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef273c1-35bc-409f-888f-0d4020d5c546 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:53:12.879698 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062506c-a395-4b24-ba33-7456e36981a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:53:12.880646 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062506c-a395-4b24-ba33-7456e36981a0 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:53:12.881037 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062506c-a395-4b24-ba33-7456e36981a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:53:12.881412 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062506c-a395-4b24-ba33-7456e36981a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:53:12.893870 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2062506c-a395-4b24-ba33-7456e36981a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:53:12.895362 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2062506c-a395-4b24-ba33-7456e36981a0 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:53:12.907368 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1125/2251] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:53:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:53:12.994944 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e49b6-2fdc-4401-ac86-c5a6d7a1c326 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:53:12.995266 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e49b6-2fdc-4401-ac86-c5a6d7a1c326 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:53:12.995553 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e49b6-2fdc-4401-ac86-c5a6d7a1c326 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:53:12.996941 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1127/2252] 10.222.0.15 () {54 vars in 921 bytes} [Wed Jul 20 14:53:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 14:53:13.002542 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f88e9-0e4b-4a77-9c55-b3f409e32a84 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:53:13.002835 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f88e9-0e4b-4a77-9c55-b3f409e32a84 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:53:13.003145 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f88e9-0e4b-4a77-9c55-b3f409e32a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:53:13.024781 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-2c5f88e9-0e4b-4a77-9c55-b3f409e32a84 None None] MFA Rules not processed for user `5d4a107ed04348b59d20d1703dbe04eb`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:53:13.026007 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-2c5f88e9-0e4b-4a77-9c55-b3f409e32a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:53:13.036975 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1126/2253] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Jul 20 14:53:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:53:13.048724 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e49b6-2fdc-4401-ac86-c5a6d7a1c326 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:53:13.066022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e49b6-2fdc-4401-ac86-c5a6d7a1c326 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:53:13.070640 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22159c7-9a3a-475b-9915-ea221868cd5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:53:13.071368 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22159c7-9a3a-475b-9915-ea221868cd5b service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:53:13.071648 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22159c7-9a3a-475b-9915-ea221868cd5b service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:53:13.071901 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22159c7-9a3a-475b-9915-ea221868cd5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:53:13.085005 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22159c7-9a3a-475b-9915-ea221868cd5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:53:13.086456 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22159c7-9a3a-475b-9915-ea221868cd5b service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:53:13.098313 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1128/2254] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:53:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:53:13.169455 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f88e9-0e4b-4a77-9c55-b3f409e32a84 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:53:13.171846 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-2c5f88e9-0e4b-4a77-9c55-b3f409e32a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:53:13.258307 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f88e9-0e4b-4a77-9c55-b3f409e32a84 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:53:13.264825 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34239412-ee9d-4078-abad-ada36bc40103 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb857f32bbc04d82bfa681bd5659765f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:53:13.265708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34239412-ee9d-4078-abad-ada36bc40103 service placement] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:53:13.266013 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34239412-ee9d-4078-abad-ada36bc40103 service placement] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:53:13.266282 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34239412-ee9d-4078-abad-ada36bc40103 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:53:13.268591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-34239412-ee9d-4078-abad-ada36bc40103 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:53:13.281498 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34239412-ee9d-4078-abad-ada36bc40103 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:53:13.283150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34239412-ee9d-4078-abad-ada36bc40103 service placement] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:53:13.293934 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1127/2255] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:53:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:53:14.728909 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22159c7-9a3a-475b-9915-ea221868cd5b service neutron] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:53:14.738296 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22159c7-9a3a-475b-9915-ea221868cd5b service neutron] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:53:14.742064 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e2cd8-78e9-4395-a0c7-69c59ea1ff0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:53:14.742716 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e2cd8-78e9-4395-a0c7-69c59ea1ff0b service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:53:14.742993 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e2cd8-78e9-4395-a0c7-69c59ea1ff0b service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:53:14.743297 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e2cd8-78e9-4395-a0c7-69c59ea1ff0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:53:14.745052 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-270e2cd8-78e9-4395-a0c7-69c59ea1ff0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:53:14.813703 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270e2cd8-78e9-4395-a0c7-69c59ea1ff0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:53:14.815181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270e2cd8-78e9-4395-a0c7-69c59ea1ff0b service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:53:14.825820 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1129/2256] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:53:14 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:53:16.187193 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34239412-ee9d-4078-abad-ada36bc40103 service placement] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:53:16.188852 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-34239412-ee9d-4078-abad-ada36bc40103 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:53:16.202982 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34239412-ee9d-4078-abad-ada36bc40103 service placement] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:53:16.207680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4c0a2-fa61-45d5-8fc2-7e72df4accc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:53:16.208342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4c0a2-fa61-45d5-8fc2-7e72df4accc2 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:53:16.208804 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4c0a2-fa61-45d5-8fc2-7e72df4accc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:53:16.209088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4c0a2-fa61-45d5-8fc2-7e72df4accc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:53:16.211207 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-b5c4c0a2-fa61-45d5-8fc2-7e72df4accc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:53:16.222763 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4c0a2-fa61-45d5-8fc2-7e72df4accc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:53:16.224319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4c0a2-fa61-45d5-8fc2-7e72df4accc2 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:53:16.234138 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1128/2257] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:53:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:53:17.335209 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41425ce0-e0b4-499c-abc3-894deb700c4a None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:53:17.336159 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41425ce0-e0b4-499c-abc3-894deb700c4a None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:53:17.336633 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41425ce0-e0b4-499c-abc3-894deb700c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:53:17.358385 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-41425ce0-e0b4-499c-abc3-894deb700c4a None None] MFA Rules not processed for user `2f8cf6d908314281a3ddc9d9513d39c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:53:17.360103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-41425ce0-e0b4-499c-abc3-894deb700c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:53:17.367550 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1130/2258] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Jul 20 14:53:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:53:17.392522 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4c0a2-fa61-45d5-8fc2-7e72df4accc2 service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:53:17.407175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4c0a2-fa61-45d5-8fc2-7e72df4accc2 service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:53:17.414427 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d9f61c-6d99-4d40-9994-c32ebaf5a3bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f4844988474d7a91133d77906611a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:53:17.415617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9f61c-6d99-4d40-9994-c32ebaf5a3bc service swift] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:53:17.416113 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9f61c-6d99-4d40-9994-c32ebaf5a3bc service swift] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:53:17.416554 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9f61c-6d99-4d40-9994-c32ebaf5a3bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:53:17.435042 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d9f61c-6d99-4d40-9994-c32ebaf5a3bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:53:17.437280 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d9f61c-6d99-4d40-9994-c32ebaf5a3bc service swift] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:53:17.455220 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1129/2259] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jul 20 14:53:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 14:53:27.354580 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=67 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:53:27.443380 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=62 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:53:27.876307 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41425ce0-e0b4-499c-abc3-894deb700c4a None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:53:27.879605 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-41425ce0-e0b4-499c-abc3-894deb700c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:53:27.901684 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41425ce0-e0b4-499c-abc3-894deb700c4a None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:53:27.908017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19a2a9f-e5da-403d-9422-0a953117a74e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:53:27.908843 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a2a9f-e5da-403d-9422-0a953117a74e service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:53:27.909260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a2a9f-e5da-403d-9422-0a953117a74e service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:53:27.909541 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a2a9f-e5da-403d-9422-0a953117a74e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:53:27.911836 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-f19a2a9f-e5da-403d-9422-0a953117a74e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:53:27.978587 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19a2a9f-e5da-403d-9422-0a953117a74e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:53:27.980696 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19a2a9f-e5da-403d-9422-0a953117a74e service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:53:27.992227 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1131/2260] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:53:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:53:37.987180 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=28 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:53:52.534184 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d9f61c-6d99-4d40-9994-c32ebaf5a3bc service swift] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:53:52.548553 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d9f61c-6d99-4d40-9994-c32ebaf5a3bc service swift] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:53:52.570274 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5435c59-705c-4311-89e8-0853f2f2b0da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:53:52.570995 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5435c59-705c-4311-89e8-0853f2f2b0da service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:53:52.571329 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5435c59-705c-4311-89e8-0853f2f2b0da service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:53:52.571606 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5435c59-705c-4311-89e8-0853f2f2b0da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:53:52.576732 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-f5435c59-705c-4311-89e8-0853f2f2b0da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:53:52.591598 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5435c59-705c-4311-89e8-0853f2f2b0da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:53:52.593326 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5435c59-705c-4311-89e8-0853f2f2b0da service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:53:52.605479 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1130/2261] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:53:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:53:54.584508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19a2a9f-e5da-403d-9422-0a953117a74e service nova] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:53:54.596563 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19a2a9f-e5da-403d-9422-0a953117a74e service nova] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:53:54.603353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ce164-ff92-487a-98f2-558150502046 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:53:54.604016 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ce164-ff92-487a-98f2-558150502046 service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:53:54.604302 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ce164-ff92-487a-98f2-558150502046 service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:53:54.604580 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ce164-ff92-487a-98f2-558150502046 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:53:54.606492 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-144ce164-ff92-487a-98f2-558150502046 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:53:54.678027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ce164-ff92-487a-98f2-558150502046 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:53:54.680653 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ce164-ff92-487a-98f2-558150502046 service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:53:54.694984 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1132/2262] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:53:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:54:02.606035 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=6 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:54:04.688058 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=28 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:54:25.109341 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5435c59-705c-4311-89e8-0853f2f2b0da service nova] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:54:25.122935 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5435c59-705c-4311-89e8-0853f2f2b0da service nova] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:54:25.131839 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a324b9-1842-42f9-b2d9-053ccedb5332 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:54:25.132509 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a324b9-1842-42f9-b2d9-053ccedb5332 service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:54:25.132796 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a324b9-1842-42f9-b2d9-053ccedb5332 service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:54:25.133100 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a324b9-1842-42f9-b2d9-053ccedb5332 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:54:25.135371 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-15a324b9-1842-42f9-b2d9-053ccedb5332 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:54:25.199091 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a324b9-1842-42f9-b2d9-053ccedb5332 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:54:25.200756 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a324b9-1842-42f9-b2d9-053ccedb5332 service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:54:25.211174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1131/2263] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:54:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:54:35.206438 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=28 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:54:37.330424 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ce164-ff92-487a-98f2-558150502046 service nova] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:54:37.343171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ce164-ff92-487a-98f2-558150502046 service nova] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:54:37.348911 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd36a94-929e-4a62-91e5-e2357fe075ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:54:37.349602 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd36a94-929e-4a62-91e5-e2357fe075ff service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:54:37.349887 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd36a94-929e-4a62-91e5-e2357fe075ff service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:54:37.350161 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd36a94-929e-4a62-91e5-e2357fe075ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:54:37.352267 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-edd36a94-929e-4a62-91e5-e2357fe075ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:54:37.413021 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd36a94-929e-4a62-91e5-e2357fe075ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:54:37.414640 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd36a94-929e-4a62-91e5-e2357fe075ff service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:54:37.425004 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1133/2264] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:54:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 14:54:47.419987 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=28 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:54:49.184297 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a324b9-1842-42f9-b2d9-053ccedb5332 service nova] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:54:49.201211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a324b9-1842-42f9-b2d9-053ccedb5332 service nova] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:54:49.207560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ff0d7-695a-4a3a-9b24-4668578d3a09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:54:49.208487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ff0d7-695a-4a3a-9b24-4668578d3a09 service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:54:49.208974 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ff0d7-695a-4a3a-9b24-4668578d3a09 service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:54:49.209610 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ff0d7-695a-4a3a-9b24-4668578d3a09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:54:49.213313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e57ff0d7-695a-4a3a-9b24-4668578d3a09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:54:49.227106 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ff0d7-695a-4a3a-9b24-4668578d3a09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:54:49.229012 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ff0d7-695a-4a3a-9b24-4668578d3a09 service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:54:49.253952 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1132/2265] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:54:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:54:49.389280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd36a94-929e-4a62-91e5-e2357fe075ff service nova] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:54:49.403994 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd36a94-929e-4a62-91e5-e2357fe075ff service nova] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:54:49.411462 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbdb0c6-c8e3-49ad-b02a-743b37fb9eb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:54:49.412366 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdb0c6-c8e3-49ad-b02a-743b37fb9eb1 service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:54:49.412831 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdb0c6-c8e3-49ad-b02a-743b37fb9eb1 service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:54:49.413294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdb0c6-c8e3-49ad-b02a-743b37fb9eb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:54:49.415870 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-acbdb0c6-c8e3-49ad-b02a-743b37fb9eb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:54:49.430078 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbdb0c6-c8e3-49ad-b02a-743b37fb9eb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:54:49.431725 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbdb0c6-c8e3-49ad-b02a-743b37fb9eb1 service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:54:49.444084 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1134/2266] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:54:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:54:51.191767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50783fb6-12c3-439d-83fa-084c6ed4c779 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:54:51.193816 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50783fb6-12c3-439d-83fa-084c6ed4c779 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:54:51.194364 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50783fb6-12c3-439d-83fa-084c6ed4c779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:54:51.213087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-50783fb6-12c3-439d-83fa-084c6ed4c779 None None] MFA Rules not processed for user `2f8cf6d908314281a3ddc9d9513d39c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:54:51.214459 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-50783fb6-12c3-439d-83fa-084c6ed4c779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:54:51.226364 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1133/2267] 10.222.0.15 () {58 vars in 967 bytes} [Wed Jul 20 14:54:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:54:51.276594 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbdb0c6-c8e3-49ad-b02a-743b37fb9eb1 service nova] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:54:51.288912 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbdb0c6-c8e3-49ad-b02a-743b37fb9eb1 service nova] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:54:51.298106 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a6fa96-0433-4737-8189-cefd72ff2f85 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f4844988474d7a91133d77906611a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:54:51.298307 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6fa96-0433-4737-8189-cefd72ff2f85 service swift] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:54:51.298586 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6fa96-0433-4737-8189-cefd72ff2f85 service swift] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:54:51.298863 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6fa96-0433-4737-8189-cefd72ff2f85 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:54:51.308559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a6fa96-0433-4737-8189-cefd72ff2f85 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:54:51.310907 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a6fa96-0433-4737-8189-cefd72ff2f85 service swift] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:54:51.320938 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1135/2268] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jul 20 14:54:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 14:55:01.207614 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=9 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:55:01.317723 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=12 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:55:09.314293 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4c0aa-2281-445f-9e41-d4347197d955 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:55:09.314830 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4c0aa-2281-445f-9e41-d4347197d955 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:55:09.314931 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4c0aa-2281-445f-9e41-d4347197d955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:55:09.333655 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-a8f4c0aa-2281-445f-9e41-d4347197d955 None None] MFA Rules not processed for user `938083ff74d84224be142a26d8e17ad6`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 14:55:09.334878 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-a8f4c0aa-2281-445f-9e41-d4347197d955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:55:09.343160 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1134/2269] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 14:55:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 14:55:19.329203 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:55:52.866226 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a6fa96-0433-4737-8189-cefd72ff2f85 service swift] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:55:52.929670 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a6fa96-0433-4737-8189-cefd72ff2f85 service swift] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:55:52.935564 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf5ae7-d1ef-45b7-ad62-18b01c17059c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb857f32bbc04d82bfa681bd5659765f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:55:52.936244 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf5ae7-d1ef-45b7-ad62-18b01c17059c service placement] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:55:52.936528 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf5ae7-d1ef-45b7-ad62-18b01c17059c service placement] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:55:52.936805 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf5ae7-d1ef-45b7-ad62-18b01c17059c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:55:52.938857 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-4abf5ae7-d1ef-45b7-ad62-18b01c17059c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:55:52.978227 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf5ae7-d1ef-45b7-ad62-18b01c17059c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:55:52.979974 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf5ae7-d1ef-45b7-ad62-18b01c17059c service placement] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:55:52.991389 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1136/2270] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:55:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:56:02.986402 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=36 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:58:50.118354 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f4c0aa-2281-445f-9e41-d4347197d955 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:58:50.146596 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f4c0aa-2281-445f-9e41-d4347197d955 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:58:50.151078 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c83e51-6bc4-455c-b816-64d72f590195 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:58:50.151752 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c83e51-6bc4-455c-b816-64d72f590195 service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:58:50.152043 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c83e51-6bc4-455c-b816-64d72f590195 service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:58:50.152321 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c83e51-6bc4-455c-b816-64d72f590195 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:58:50.154410 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-01c83e51-6bc4-455c-b816-64d72f590195 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:58:50.215144 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c83e51-6bc4-455c-b816-64d72f590195 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:58:50.216600 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c83e51-6bc4-455c-b816-64d72f590195 service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:58:50.226066 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1135/2271] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:58:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:58:50.466380 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf5ae7-d1ef-45b7-ad62-18b01c17059c service placement] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:58:50.488034 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf5ae7-d1ef-45b7-ad62-18b01c17059c service placement] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:58:50.492362 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c4994f-e277-48ee-b489-4f48589e6647 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:58:50.493007 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c4994f-e277-48ee-b489-4f48589e6647 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:58:50.493314 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c4994f-e277-48ee-b489-4f48589e6647 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:58:50.493611 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c4994f-e277-48ee-b489-4f48589e6647 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:58:50.501471 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c4994f-e277-48ee-b489-4f48589e6647 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:58:50.502893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c4994f-e277-48ee-b489-4f48589e6647 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:58:50.511078 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1137/2272] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:58:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:58:50.866968 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c83e51-6bc4-455c-b816-64d72f590195 service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:58:50.869347 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-01c83e51-6bc4-455c-b816-64d72f590195 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 14:58:50.899430 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c83e51-6bc4-455c-b816-64d72f590195 service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:58:50.903700 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33c6a2-fe89-4eff-b3f3-1e7c6012559c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:58:50.904362 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33c6a2-fe89-4eff-b3f3-1e7c6012559c service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:58:50.904644 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33c6a2-fe89-4eff-b3f3-1e7c6012559c service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:58:50.904909 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33c6a2-fe89-4eff-b3f3-1e7c6012559c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:58:50.923088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f33c6a2-fe89-4eff-b3f3-1e7c6012559c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:58:50.924527 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f33c6a2-fe89-4eff-b3f3-1e7c6012559c service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:58:50.946551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1136/2273] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:58:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:58:50.995350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c4994f-e277-48ee-b489-4f48589e6647 service neutron] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:58:51.005789 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c4994f-e277-48ee-b489-4f48589e6647 service neutron] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:58:51.010096 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8420249-f8e2-4a61-b167-bec72f8a6999 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:58:51.010878 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8420249-f8e2-4a61-b167-bec72f8a6999 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:58:51.011257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8420249-f8e2-4a61-b167-bec72f8a6999 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:58:51.011600 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8420249-f8e2-4a61-b167-bec72f8a6999 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:58:51.013568 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33c6a2-fe89-4eff-b3f3-1e7c6012559c service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:58:51.024600 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8420249-f8e2-4a61-b167-bec72f8a6999 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:58:51.026217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8420249-f8e2-4a61-b167-bec72f8a6999 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:58:51.029105 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33c6a2-fe89-4eff-b3f3-1e7c6012559c service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:58:51.036785 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e71be1-2094-4a60-995f-eb2731af2940 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:58:51.037448 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e71be1-2094-4a60-995f-eb2731af2940 service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:58:51.037741 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e71be1-2094-4a60-995f-eb2731af2940 service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:58:51.038654 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e71be1-2094-4a60-995f-eb2731af2940 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:58:51.038803 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1138/2274] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:58:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:58:51.045567 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8420249-f8e2-4a61-b167-bec72f8a6999 service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:58:51.056239 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8420249-f8e2-4a61-b167-bec72f8a6999 service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:58:51.057276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e71be1-2094-4a60-995f-eb2731af2940 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:58:51.059005 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e71be1-2094-4a60-995f-eb2731af2940 service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:58:51.062145 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5129c1-d5bb-4ca2-b9db-228150368074 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:58:51.063011 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5129c1-d5bb-4ca2-b9db-228150368074 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:58:51.063383 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5129c1-d5bb-4ca2-b9db-228150368074 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:58:51.063868 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5129c1-d5bb-4ca2-b9db-228150368074 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:58:51.073741 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5129c1-d5bb-4ca2-b9db-228150368074 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:58:51.075303 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5129c1-d5bb-4ca2-b9db-228150368074 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:58:51.093249 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1137/2275] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:58:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 14:58:51.110654 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1139/2276] 10.222.0.15 () {60 vars in 1396 bytes} [Wed Jul 20 14:58:51 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3254 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 14:58:51.223159 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e71be1-2094-4a60-995f-eb2731af2940 service nova] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 14:58:51.233419 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e71be1-2094-4a60-995f-eb2731af2940 service nova] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 14:58:51.242835 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955bae08-f104-46a4-97c6-c6259a99d016 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 14:58:51.243538 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bae08-f104-46a4-97c6-c6259a99d016 service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 14:58:51.243826 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bae08-f104-46a4-97c6-c6259a99d016 service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 14:58:51.244101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bae08-f104-46a4-97c6-c6259a99d016 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 14:58:51.253883 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955bae08-f104-46a4-97c6-c6259a99d016 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 14:58:51.261941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955bae08-f104-46a4-97c6-c6259a99d016 service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 14:58:51.272469 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1138/2277] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 14:58:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 14:59:01.107942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=29 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:59:01.268025 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=71 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:00:50.264609 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5129c1-d5bb-4ca2-b9db-228150368074 service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:00:50.278171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5129c1-d5bb-4ca2-b9db-228150368074 service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:00:50.282636 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49bac1c-38cf-40ce-9005-48b0542694dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:00:50.283332 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bac1c-38cf-40ce-9005-48b0542694dc service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:00:50.283615 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bac1c-38cf-40ce-9005-48b0542694dc service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:00:50.286068 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bac1c-38cf-40ce-9005-48b0542694dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:00:50.294284 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49bac1c-38cf-40ce-9005-48b0542694dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:00:50.295763 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49bac1c-38cf-40ce-9005-48b0542694dc service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:00:50.325424 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1140/2278] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:00:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:00:51.411962 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955bae08-f104-46a4-97c6-c6259a99d016 service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:00:51.424814 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955bae08-f104-46a4-97c6-c6259a99d016 service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:00:51.428950 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7dbd5-0b67-411d-85ed-553127641c96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:00:51.430153 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7dbd5-0b67-411d-85ed-553127641c96 service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:00:51.430153 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7dbd5-0b67-411d-85ed-553127641c96 service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:00:51.430153 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7dbd5-0b67-411d-85ed-553127641c96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:00:51.440624 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7dbd5-0b67-411d-85ed-553127641c96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:00:51.442070 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7dbd5-0b67-411d-85ed-553127641c96 service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:00:51.479184 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1139/2279] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:00:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:00:51.560090 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49bac1c-38cf-40ce-9005-48b0542694dc service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:00:51.570634 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49bac1c-38cf-40ce-9005-48b0542694dc service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:00:51.575058 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a80a5-e1b9-4574-b134-12f2d215f7c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:00:51.575741 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a80a5-e1b9-4574-b134-12f2d215f7c1 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:00:51.576106 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a80a5-e1b9-4574-b134-12f2d215f7c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:00:51.576305 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a80a5-e1b9-4574-b134-12f2d215f7c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:00:51.586618 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a80a5-e1b9-4574-b134-12f2d215f7c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:00:51.588156 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a80a5-e1b9-4574-b134-12f2d215f7c1 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:00:51.623482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1141/2280] 10.222.0.15 () {60 vars in 1396 bytes} [Wed Jul 20 15:00:51 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3252 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:00:59.749517 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7dbd5-0b67-411d-85ed-553127641c96 service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:00:59.751239 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-17c7dbd5-0b67-411d-85ed-553127641c96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:00:59.768237 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7dbd5-0b67-411d-85ed-553127641c96 service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:00:59.775375 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452b6ae-36a6-4955-a255-032aedfdfe3f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb857f32bbc04d82bfa681bd5659765f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:00:59.776064 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452b6ae-36a6-4955-a255-032aedfdfe3f service placement] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:00:59.776376 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452b6ae-36a6-4955-a255-032aedfdfe3f service placement] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:00:59.776688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452b6ae-36a6-4955-a255-032aedfdfe3f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:00:59.786546 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f452b6ae-36a6-4955-a255-032aedfdfe3f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:00:59.788194 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f452b6ae-36a6-4955-a255-032aedfdfe3f service placement] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:00:59.799262 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1140/2281] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:00:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:01:01.620251 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=26 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:01:09.794788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=19 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:03:57.425801 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a80a5-e1b9-4574-b134-12f2d215f7c1 service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:03:57.427768 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-858a80a5-e1b9-4574-b134-12f2d215f7c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:03:57.503628 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a80a5-e1b9-4574-b134-12f2d215f7c1 service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:03:57.507804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34721a58-8c2f-4e6b-b2bd-8b9c9fed4e4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:03:57.508456 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34721a58-8c2f-4e6b-b2bd-8b9c9fed4e4c service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:03:57.508743 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34721a58-8c2f-4e6b-b2bd-8b9c9fed4e4c service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:03:57.509019 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34721a58-8c2f-4e6b-b2bd-8b9c9fed4e4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:03:57.511220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-34721a58-8c2f-4e6b-b2bd-8b9c9fed4e4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:03:57.522341 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34721a58-8c2f-4e6b-b2bd-8b9c9fed4e4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:03:57.523832 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34721a58-8c2f-4e6b-b2bd-8b9c9fed4e4c service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:03:57.532127 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1142/2282] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:03:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:04:07.528621 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=31 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:04:50.510743 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452b6ae-36a6-4955-a255-032aedfdfe3f service placement] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:04:50.512551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-f452b6ae-36a6-4955-a255-032aedfdfe3f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:04:50.570874 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452b6ae-36a6-4955-a255-032aedfdfe3f service placement] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:04:50.575351 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac07acb-c66b-4f7a-bad9-bdab519b675e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:04:50.576020 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac07acb-c66b-4f7a-bad9-bdab519b675e service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:04:50.576307 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac07acb-c66b-4f7a-bad9-bdab519b675e service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:04:50.576582 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac07acb-c66b-4f7a-bad9-bdab519b675e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:04:50.578823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-fac07acb-c66b-4f7a-bad9-bdab519b675e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:04:50.665625 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac07acb-c66b-4f7a-bad9-bdab519b675e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:04:50.667102 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac07acb-c66b-4f7a-bad9-bdab519b675e service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:04:50.686167 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1141/2283] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:04:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:04:50.716131 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34721a58-8c2f-4e6b-b2bd-8b9c9fed4e4c service neutron] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:04:50.730368 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34721a58-8c2f-4e6b-b2bd-8b9c9fed4e4c service neutron] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:04:50.736756 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf5e520-1cba-4dda-bda1-aabfa384bdd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:04:50.737693 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5e520-1cba-4dda-bda1-aabfa384bdd2 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:04:50.738047 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5e520-1cba-4dda-bda1-aabfa384bdd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:04:50.738397 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5e520-1cba-4dda-bda1-aabfa384bdd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:04:50.749956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf5e520-1cba-4dda-bda1-aabfa384bdd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:04:50.751449 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf5e520-1cba-4dda-bda1-aabfa384bdd2 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:04:50.785246 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1143/2284] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:04:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:04:52.100723 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac07acb-c66b-4f7a-bad9-bdab519b675e service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:04:52.111880 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac07acb-c66b-4f7a-bad9-bdab519b675e service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:04:52.116772 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faae572-2661-49d6-bd61-87e32a0482d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:04:52.117434 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faae572-2661-49d6-bd61-87e32a0482d3 service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:04:52.117714 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faae572-2661-49d6-bd61-87e32a0482d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:04:52.117991 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faae572-2661-49d6-bd61-87e32a0482d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:04:52.120227 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-9faae572-2661-49d6-bd61-87e32a0482d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:04:52.211330 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faae572-2661-49d6-bd61-87e32a0482d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:04:52.212773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faae572-2661-49d6-bd61-87e32a0482d3 service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:04:52.218603 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:04:52.224654 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1142/2285] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:04:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:04:52.279608 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf5e520-1cba-4dda-bda1-aabfa384bdd2 service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:04:52.288687 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf5e520-1cba-4dda-bda1-aabfa384bdd2 service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:04:52.293208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb23ea-049f-485e-ba94-98781d1c0ff7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:04:52.293874 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb23ea-049f-485e-ba94-98781d1c0ff7 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:04:52.294162 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb23ea-049f-485e-ba94-98781d1c0ff7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:04:52.294444 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb23ea-049f-485e-ba94-98781d1c0ff7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:04:52.296494 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-25cb23ea-049f-485e-ba94-98781d1c0ff7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:04:52.310940 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb23ea-049f-485e-ba94-98781d1c0ff7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:04:52.312418 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb23ea-049f-485e-ba94-98781d1c0ff7 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:04:52.324410 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1144/2286] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:04:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:04:52.331645 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faae572-2661-49d6-bd61-87e32a0482d3 service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:04:52.340805 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faae572-2661-49d6-bd61-87e32a0482d3 service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:04:52.345075 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0459d84-df37-4c34-ab60-be6f46cd4a20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:04:52.345692 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0459d84-df37-4c34-ab60-be6f46cd4a20 service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:04:52.345981 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0459d84-df37-4c34-ab60-be6f46cd4a20 service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:04:52.346255 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0459d84-df37-4c34-ab60-be6f46cd4a20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:04:52.348461 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e0459d84-df37-4c34-ab60-be6f46cd4a20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:04:52.423832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0459d84-df37-4c34-ab60-be6f46cd4a20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:04:52.425293 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0459d84-df37-4c34-ab60-be6f46cd4a20 service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:04:52.434596 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1143/2287] 10.222.0.15 () {60 vars in 1396 bytes} [Wed Jul 20 15:04:52 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3254 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:05:02.318910 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=19 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:05:02.430428 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=33 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:06:04.747589 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb23ea-049f-485e-ba94-98781d1c0ff7 service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:06:04.749341 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-25cb23ea-049f-485e-ba94-98781d1c0ff7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:06:04.906319 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb23ea-049f-485e-ba94-98781d1c0ff7 service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:06:04.911856 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87381442-7278-48a1-b94d-a6612bb15913 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb857f32bbc04d82bfa681bd5659765f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:06:04.912510 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87381442-7278-48a1-b94d-a6612bb15913 service placement] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:06:04.912793 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87381442-7278-48a1-b94d-a6612bb15913 service placement] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:06:04.913070 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87381442-7278-48a1-b94d-a6612bb15913 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:06:04.915074 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-87381442-7278-48a1-b94d-a6612bb15913 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:06:04.927443 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87381442-7278-48a1-b94d-a6612bb15913 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:06:04.929191 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87381442-7278-48a1-b94d-a6612bb15913 service placement] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:06:04.940102 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1145/2288] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:06:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:06:14.935290 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=61 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:06:50.821767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0459d84-df37-4c34-ab60-be6f46cd4a20 service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:06:50.838963 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0459d84-df37-4c34-ab60-be6f46cd4a20 service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:06:50.843636 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af2a5f-921a-40cb-8aa2-960e8ba1a8e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:06:50.844294 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af2a5f-921a-40cb-8aa2-960e8ba1a8e1 service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:06:50.844569 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af2a5f-921a-40cb-8aa2-960e8ba1a8e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:06:50.844850 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af2a5f-921a-40cb-8aa2-960e8ba1a8e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:06:50.846998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-f7af2a5f-921a-40cb-8aa2-960e8ba1a8e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:06:50.922524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7af2a5f-921a-40cb-8aa2-960e8ba1a8e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:06:50.924010 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7af2a5f-921a-40cb-8aa2-960e8ba1a8e1 service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:06:50.933405 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1144/2289] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:06:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:06:51.000659 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87381442-7278-48a1-b94d-a6612bb15913 service placement] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:06:51.010844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87381442-7278-48a1-b94d-a6612bb15913 service placement] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:06:51.015785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9389e3-d01b-4019-9c48-7fd3c3cfc266 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:06:51.016445 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9389e3-d01b-4019-9c48-7fd3c3cfc266 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:06:51.016731 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9389e3-d01b-4019-9c48-7fd3c3cfc266 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:06:51.017007 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9389e3-d01b-4019-9c48-7fd3c3cfc266 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:06:51.019198 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-db9389e3-d01b-4019-9c48-7fd3c3cfc266 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:06:51.094560 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9389e3-d01b-4019-9c48-7fd3c3cfc266 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:06:51.095948 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9389e3-d01b-4019-9c48-7fd3c3cfc266 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:06:51.105372 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1146/2290] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:06:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:06:52.579858 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af2a5f-921a-40cb-8aa2-960e8ba1a8e1 service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:06:52.589277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af2a5f-921a-40cb-8aa2-960e8ba1a8e1 service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:06:52.593936 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920a9e8-c65c-40ee-94ab-5563de3e384b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:06:52.594606 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920a9e8-c65c-40ee-94ab-5563de3e384b service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:06:52.594887 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920a9e8-c65c-40ee-94ab-5563de3e384b service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:06:52.595192 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920a9e8-c65c-40ee-94ab-5563de3e384b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:06:52.597386 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-8920a9e8-c65c-40ee-94ab-5563de3e384b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:06:52.679953 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8920a9e8-c65c-40ee-94ab-5563de3e384b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:06:52.681402 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8920a9e8-c65c-40ee-94ab-5563de3e384b service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:06:52.693545 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1145/2291] 10.222.0.15 () {60 vars in 1396 bytes} [Wed Jul 20 15:06:52 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3252 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:06:52.744465 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9389e3-d01b-4019-9c48-7fd3c3cfc266 service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:06:52.746257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-db9389e3-d01b-4019-9c48-7fd3c3cfc266 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:06:52.764192 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9389e3-d01b-4019-9c48-7fd3c3cfc266 service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:06:52.773252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b855b93d-e6ff-47ea-b440-31b3713c1009 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:06:52.773252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855b93d-e6ff-47ea-b440-31b3713c1009 service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:06:52.773252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855b93d-e6ff-47ea-b440-31b3713c1009 service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:06:52.778270 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855b93d-e6ff-47ea-b440-31b3713c1009 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:06:52.778270 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-b855b93d-e6ff-47ea-b440-31b3713c1009 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:06:52.874500 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b855b93d-e6ff-47ea-b440-31b3713c1009 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:06:52.876923 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b855b93d-e6ff-47ea-b440-31b3713c1009 service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:06:52.887838 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1147/2292] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:06:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:07:02.687478 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=69 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:07:02.882599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=66 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:09:01.414225 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920a9e8-c65c-40ee-94ab-5563de3e384b service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:09:01.438875 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920a9e8-c65c-40ee-94ab-5563de3e384b service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:09:01.446352 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255c653-2aa2-4e86-8e92-d6d00a586a6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:09:01.447511 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255c653-2aa2-4e86-8e92-d6d00a586a6f service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:09:01.447803 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255c653-2aa2-4e86-8e92-d6d00a586a6f service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:09:01.448116 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255c653-2aa2-4e86-8e92-d6d00a586a6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:09:01.457020 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1255c653-2aa2-4e86-8e92-d6d00a586a6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:09:01.458489 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1255c653-2aa2-4e86-8e92-d6d00a586a6f service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:09:01.466754 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1146/2293] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:09:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:09:11.463150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=12 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:10:51.331942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b855b93d-e6ff-47ea-b440-31b3713c1009 service nova] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:10:51.361347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b855b93d-e6ff-47ea-b440-31b3713c1009 service nova] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:10:51.365949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f2e313-e71e-4dff-933d-42d0e5e5db51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:10:51.366633 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2e313-e71e-4dff-933d-42d0e5e5db51 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:10:51.366913 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2e313-e71e-4dff-933d-42d0e5e5db51 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:10:51.367234 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2e313-e71e-4dff-933d-42d0e5e5db51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:10:51.385697 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f2e313-e71e-4dff-933d-42d0e5e5db51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:10:51.387159 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f2e313-e71e-4dff-933d-42d0e5e5db51 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:10:51.409588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1148/2294] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:10:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:10:51.459796 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255c653-2aa2-4e86-8e92-d6d00a586a6f service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:10:51.469246 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255c653-2aa2-4e86-8e92-d6d00a586a6f service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:10:51.472968 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c04da55-617d-41cb-93bc-7be620cb880a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:10:51.473615 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04da55-617d-41cb-93bc-7be620cb880a service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:10:51.473926 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04da55-617d-41cb-93bc-7be620cb880a service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:10:51.474203 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04da55-617d-41cb-93bc-7be620cb880a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:10:51.485879 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c04da55-617d-41cb-93bc-7be620cb880a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:10:51.487377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c04da55-617d-41cb-93bc-7be620cb880a service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:10:51.498236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1147/2295] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:10:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:10:51.505622 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f2e313-e71e-4dff-933d-42d0e5e5db51 service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:10:51.514933 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f2e313-e71e-4dff-933d-42d0e5e5db51 service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:10:51.518632 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9393c2bf-9c89-4adc-a9c2-97d6578ecc7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:10:51.519304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393c2bf-9c89-4adc-a9c2-97d6578ecc7e service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:10:51.519589 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393c2bf-9c89-4adc-a9c2-97d6578ecc7e service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:10:51.519864 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393c2bf-9c89-4adc-a9c2-97d6578ecc7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:10:51.528657 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9393c2bf-9c89-4adc-a9c2-97d6578ecc7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:10:51.530085 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9393c2bf-9c89-4adc-a9c2-97d6578ecc7e service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:10:51.562939 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1149/2296] 10.222.0.15 () {60 vars in 1396 bytes} [Wed Jul 20 15:10:51 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3254 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:10:53.271536 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c04da55-617d-41cb-93bc-7be620cb880a service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:10:53.273408 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-9c04da55-617d-41cb-93bc-7be620cb880a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:10:53.288537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c04da55-617d-41cb-93bc-7be620cb880a service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:10:53.294836 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f071008-9962-4023-b01b-5c13dcbcc247 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:10:53.295264 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f071008-9962-4023-b01b-5c13dcbcc247 service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:10:53.295550 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f071008-9962-4023-b01b-5c13dcbcc247 service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:10:53.295833 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f071008-9962-4023-b01b-5c13dcbcc247 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:10:53.297825 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-3f071008-9962-4023-b01b-5c13dcbcc247 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:10:53.378149 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f071008-9962-4023-b01b-5c13dcbcc247 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:10:53.379846 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f071008-9962-4023-b01b-5c13dcbcc247 service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:10:53.390113 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1148/2297] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:10:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:10:55.323214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9393c2bf-9c89-4adc-a9c2-97d6578ecc7e service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:10:55.335551 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9393c2bf-9c89-4adc-a9c2-97d6578ecc7e service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:10:55.340988 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da69eb2f-def0-40b3-ab00-e6dcf24918a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:10:55.341635 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69eb2f-def0-40b3-ab00-e6dcf24918a4 service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:10:55.341972 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69eb2f-def0-40b3-ab00-e6dcf24918a4 service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:10:55.342246 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69eb2f-def0-40b3-ab00-e6dcf24918a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:10:55.350572 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da69eb2f-def0-40b3-ab00-e6dcf24918a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:10:55.352255 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da69eb2f-def0-40b3-ab00-e6dcf24918a4 service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:10:55.361508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1150/2298] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:10:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:10:55.504203 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f071008-9962-4023-b01b-5c13dcbcc247 service nova] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:10:55.517317 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f071008-9962-4023-b01b-5c13dcbcc247 service nova] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:10:55.527741 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad88a7-bc59-463d-9e43-fc4652901e6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:10:55.528467 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad88a7-bc59-463d-9e43-fc4652901e6e service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:10:55.528758 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad88a7-bc59-463d-9e43-fc4652901e6e service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:10:55.529036 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad88a7-bc59-463d-9e43-fc4652901e6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:10:55.531072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-ccad88a7-bc59-463d-9e43-fc4652901e6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:10:55.543826 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccad88a7-bc59-463d-9e43-fc4652901e6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:10:55.545420 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccad88a7-bc59-463d-9e43-fc4652901e6e service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:10:55.556301 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1149/2299] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:10:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:10:56.726731 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da69eb2f-def0-40b3-ab00-e6dcf24918a4 service nova] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:10:56.740278 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da69eb2f-def0-40b3-ab00-e6dcf24918a4 service nova] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:10:56.747010 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4ed86-8a4b-4ca6-848e-c61114d136ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:10:56.747723 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4ed86-8a4b-4ca6-848e-c61114d136ec service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:10:56.748007 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4ed86-8a4b-4ca6-848e-c61114d136ec service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:10:56.748281 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4ed86-8a4b-4ca6-848e-c61114d136ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:10:56.750295 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-90d4ed86-8a4b-4ca6-848e-c61114d136ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:10:56.762861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d4ed86-8a4b-4ca6-848e-c61114d136ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:10:56.764472 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d4ed86-8a4b-4ca6-848e-c61114d136ec service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:10:56.774685 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1151/2300] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:10:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:11:05.552377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=45 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:06.770434 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=44 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:13.764350 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad88a7-bc59-463d-9e43-fc4652901e6e service nova] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:13.776569 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad88a7-bc59-463d-9e43-fc4652901e6e service nova] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:13.782782 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44e712-4add-4c05-9e8e-bad63fac1440 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb857f32bbc04d82bfa681bd5659765f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:13.783469 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44e712-4add-4c05-9e8e-bad63fac1440 service placement] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:13.783722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44e712-4add-4c05-9e8e-bad63fac1440 service placement] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:13.784027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44e712-4add-4c05-9e8e-bad63fac1440 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:13.792900 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f44e712-4add-4c05-9e8e-bad63fac1440 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:13.794487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f44e712-4add-4c05-9e8e-bad63fac1440 service placement] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:13.804988 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1150/2301] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:11:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:11:23.800592 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=6 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:51.130572 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4ed86-8a4b-4ca6-848e-c61114d136ec service nova] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:51.143151 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4ed86-8a4b-4ca6-848e-c61114d136ec service nova] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:51.152453 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3893b7b-7768-459a-a4af-dd738e49101d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:51.153038 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3893b7b-7768-459a-a4af-dd738e49101d service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:51.153326 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3893b7b-7768-459a-a4af-dd738e49101d service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:51.153595 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3893b7b-7768-459a-a4af-dd738e49101d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:51.162677 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3893b7b-7768-459a-a4af-dd738e49101d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:51.164291 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3893b7b-7768-459a-a4af-dd738e49101d service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:51.193656 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1152/2302] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:11:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:11:53.154470 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44e712-4add-4c05-9e8e-bad63fac1440 service placement] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:53.174054 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44e712-4add-4c05-9e8e-bad63fac1440 service placement] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:53.180866 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af1982-e502-4550-9d14-b240cf8bc8a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:53.181482 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af1982-e502-4550-9d14-b240cf8bc8a9 service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:53.181764 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af1982-e502-4550-9d14-b240cf8bc8a9 service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:53.182044 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af1982-e502-4550-9d14-b240cf8bc8a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:53.192081 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af1982-e502-4550-9d14-b240cf8bc8a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:53.194523 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af1982-e502-4550-9d14-b240cf8bc8a9 service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:53.226474 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1151/2303] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:11:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:12:01.190929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=13 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:03.223201 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=13 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:51.688868 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3893b7b-7768-459a-a4af-dd738e49101d service nova] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:51.699696 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3893b7b-7768-459a-a4af-dd738e49101d service nova] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:51.704057 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b573e-1766-4e4d-b890-a3deed215103 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:51.704812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b573e-1766-4e4d-b890-a3deed215103 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:51.705156 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b573e-1766-4e4d-b890-a3deed215103 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:51.705501 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b573e-1766-4e4d-b890-a3deed215103 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:51.714713 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b573e-1766-4e4d-b890-a3deed215103 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:51.716290 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b573e-1766-4e4d-b890-a3deed215103 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:51.743976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1153/2304] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:12:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:12:51.778073 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af1982-e502-4550-9d14-b240cf8bc8a9 service nova] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:51.789321 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af1982-e502-4550-9d14-b240cf8bc8a9 service nova] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:51.795392 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7755a1c-e45b-4df0-90e5-2e5007c2fc36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:51.796088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7755a1c-e45b-4df0-90e5-2e5007c2fc36 service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:51.796356 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7755a1c-e45b-4df0-90e5-2e5007c2fc36 service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:51.796629 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7755a1c-e45b-4df0-90e5-2e5007c2fc36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:51.804887 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7755a1c-e45b-4df0-90e5-2e5007c2fc36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:51.806555 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7755a1c-e45b-4df0-90e5-2e5007c2fc36 service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:51.834872 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1152/2305] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:12:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:13:01.741423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=13 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:01.832203 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=13 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:05.144961 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b573e-1766-4e4d-b890-a3deed215103 service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:05.146638 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-f65b573e-1766-4e4d-b890-a3deed215103 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:14:05.218087 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b573e-1766-4e4d-b890-a3deed215103 service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:05.221970 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364aeff3-aebc-4cb4-b28c-aee9c51085a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:05.222624 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364aeff3-aebc-4cb4-b28c-aee9c51085a0 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:05.222907 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364aeff3-aebc-4cb4-b28c-aee9c51085a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:05.223217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364aeff3-aebc-4cb4-b28c-aee9c51085a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:05.225248 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-364aeff3-aebc-4cb4-b28c-aee9c51085a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:14:05.235291 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364aeff3-aebc-4cb4-b28c-aee9c51085a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:05.236717 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364aeff3-aebc-4cb4-b28c-aee9c51085a0 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:05.244984 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1154/2306] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:14:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:14:15.241721 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=31 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:51.754806 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7755a1c-e45b-4df0-90e5-2e5007c2fc36 service nova] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:51.767431 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7755a1c-e45b-4df0-90e5-2e5007c2fc36 service nova] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:51.774061 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a1a55b-3133-433a-9de0-4fdf6f651bef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:51.774746 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1a55b-3133-433a-9de0-4fdf6f651bef service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:51.775028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1a55b-3133-433a-9de0-4fdf6f651bef service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:51.775336 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1a55b-3133-433a-9de0-4fdf6f651bef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:51.777425 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-57a1a55b-3133-433a-9de0-4fdf6f651bef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:15:51.871663 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a1a55b-3133-433a-9de0-4fdf6f651bef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:51.873074 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a1a55b-3133-433a-9de0-4fdf6f651bef service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:51.884066 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1153/2307] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:15:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:16:01.879714 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=39 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:17.742547 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364aeff3-aebc-4cb4-b28c-aee9c51085a0 service neutron] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:17.744342 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-364aeff3-aebc-4cb4-b28c-aee9c51085a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:16:17.899834 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364aeff3-aebc-4cb4-b28c-aee9c51085a0 service neutron] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:17.906094 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a18e6-e2a0-4ddb-a821-70c3453ec27b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb857f32bbc04d82bfa681bd5659765f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:17.906750 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a18e6-e2a0-4ddb-a821-70c3453ec27b service placement] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:17.907077 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a18e6-e2a0-4ddb-a821-70c3453ec27b service placement] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:17.907365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a18e6-e2a0-4ddb-a821-70c3453ec27b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:17.909522 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-d37a18e6-e2a0-4ddb-a821-70c3453ec27b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:16:17.921626 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37a18e6-e2a0-4ddb-a821-70c3453ec27b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:17.923245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37a18e6-e2a0-4ddb-a821-70c3453ec27b service placement] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:17.933603 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1155/2308] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:16:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:16:27.929341 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=58 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:52.452176 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a1a55b-3133-433a-9de0-4fdf6f651bef service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:52.455284 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-57a1a55b-3133-433a-9de0-4fdf6f651bef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:16:52.557474 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a1a55b-3133-433a-9de0-4fdf6f651bef service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:52.561459 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e79500a-7bb6-436c-aabd-4f11e55974a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:52.562153 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79500a-7bb6-436c-aabd-4f11e55974a1 service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:52.562452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79500a-7bb6-436c-aabd-4f11e55974a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:52.562744 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79500a-7bb6-436c-aabd-4f11e55974a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:52.564797 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-0e79500a-7bb6-436c-aabd-4f11e55974a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:16:52.579498 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79500a-7bb6-436c-aabd-4f11e55974a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:52.580936 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79500a-7bb6-436c-aabd-4f11e55974a1 service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:52.592169 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1154/2309] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:16:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:16:52.599698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a18e6-e2a0-4ddb-a821-70c3453ec27b service placement] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:52.616964 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a18e6-e2a0-4ddb-a821-70c3453ec27b service placement] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:52.621733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f319130-ddbc-4adb-b8e0-c8d8e8dd55ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:52.622392 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f319130-ddbc-4adb-b8e0-c8d8e8dd55ae service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:52.622678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f319130-ddbc-4adb-b8e0-c8d8e8dd55ae service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:52.622958 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f319130-ddbc-4adb-b8e0-c8d8e8dd55ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:52.632227 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f319130-ddbc-4adb-b8e0-c8d8e8dd55ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:52.633666 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f319130-ddbc-4adb-b8e0-c8d8e8dd55ae service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:52.642698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1156/2310] 10.222.0.15 () {60 vars in 1396 bytes} [Wed Jul 20 15:16:52 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:16:52.693736 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e79500a-7bb6-436c-aabd-4f11e55974a1 service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:52.703913 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e79500a-7bb6-436c-aabd-4f11e55974a1 service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:52.710857 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d71a5-463a-49b1-877f-422abb8cb733 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:52.711674 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d71a5-463a-49b1-877f-422abb8cb733 service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:52.712144 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d71a5-463a-49b1-877f-422abb8cb733 service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:52.712482 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d71a5-463a-49b1-877f-422abb8cb733 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:52.721599 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21d71a5-463a-49b1-877f-422abb8cb733 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:52.723148 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21d71a5-463a-49b1-877f-422abb8cb733 service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:52.729332 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f319130-ddbc-4adb-b8e0-c8d8e8dd55ae service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:52.737700 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f319130-ddbc-4adb-b8e0-c8d8e8dd55ae service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:52.741445 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5404bc-59c4-4387-ba87-5a403df714e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:52.742063 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5404bc-59c4-4387-ba87-5a403df714e1 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:52.742346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5404bc-59c4-4387-ba87-5a403df714e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:52.742648 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5404bc-59c4-4387-ba87-5a403df714e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:52.744560 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-0f5404bc-59c4-4387-ba87-5a403df714e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:16:52.752488 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1155/2311] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:16:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:16:52.829588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5404bc-59c4-4387-ba87-5a403df714e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:52.831017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5404bc-59c4-4387-ba87-5a403df714e1 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:52.850740 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1157/2312] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:16:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:16:52.900234 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d71a5-463a-49b1-877f-422abb8cb733 service nova] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:52.914842 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d71a5-463a-49b1-877f-422abb8cb733 service nova] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:52.919850 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3414d2c-f800-4040-9389-d0d88bcb3311 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:52.920500 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3414d2c-f800-4040-9389-d0d88bcb3311 service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:52.920788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3414d2c-f800-4040-9389-d0d88bcb3311 service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:52.921068 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3414d2c-f800-4040-9389-d0d88bcb3311 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:52.923264 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e3414d2c-f800-4040-9389-d0d88bcb3311 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:16:53.001943 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3414d2c-f800-4040-9389-d0d88bcb3311 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:53.003331 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3414d2c-f800-4040-9389-d0d88bcb3311 service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:53.020904 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1156/2313] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:16:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:16:53.258900 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5404bc-59c4-4387-ba87-5a403df714e1 service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:53.268125 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5404bc-59c4-4387-ba87-5a403df714e1 service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:53.272815 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252cb100-e26d-4a66-b308-a4fd6c911750 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:53.273463 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cb100-e26d-4a66-b308-a4fd6c911750 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:53.273744 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cb100-e26d-4a66-b308-a4fd6c911750 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:53.274020 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cb100-e26d-4a66-b308-a4fd6c911750 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:53.276151 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-252cb100-e26d-4a66-b308-a4fd6c911750 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:16:53.358454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252cb100-e26d-4a66-b308-a4fd6c911750 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:53.360901 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252cb100-e26d-4a66-b308-a4fd6c911750 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:53.372509 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1158/2314] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:16:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:16:57.773616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3414d2c-f800-4040-9389-d0d88bcb3311 service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:57.776242 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e3414d2c-f800-4040-9389-d0d88bcb3311 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:16:57.790017 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3414d2c-f800-4040-9389-d0d88bcb3311 service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:57.793919 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bdb6f3-bdc2-4117-ab1c-ddf9646a0ac6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:57.794573 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdb6f3-bdc2-4117-ab1c-ddf9646a0ac6 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:57.794859 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdb6f3-bdc2-4117-ab1c-ddf9646a0ac6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:57.795359 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdb6f3-bdc2-4117-ab1c-ddf9646a0ac6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:57.797557 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-f4bdb6f3-bdc2-4117-ab1c-ddf9646a0ac6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:16:57.820699 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252cb100-e26d-4a66-b308-a4fd6c911750 service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:57.830567 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252cb100-e26d-4a66-b308-a4fd6c911750 service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:57.834824 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561a10d-5758-4a3b-99eb-bbe68266df8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:57.835877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561a10d-5758-4a3b-99eb-bbe68266df8b service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:57.836361 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561a10d-5758-4a3b-99eb-bbe68266df8b service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:57.836638 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561a10d-5758-4a3b-99eb-bbe68266df8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:57.837825 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:57.838469 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-6561a10d-5758-4a3b-99eb-bbe68266df8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:16:57.880454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:57.904826 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bdb6f3-bdc2-4117-ab1c-ddf9646a0ac6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:57.906336 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bdb6f3-bdc2-4117-ab1c-ddf9646a0ac6 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:57.914062 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561a10d-5758-4a3b-99eb-bbe68266df8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:57.915537 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561a10d-5758-4a3b-99eb-bbe68266df8b service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:57.925289 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1157/2315] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:16:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:16:57.935931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1159/2316] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:16:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:17:02.772002 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bdb6f3-bdc2-4117-ab1c-ddf9646a0ac6 service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:17:02.775645 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-f4bdb6f3-bdc2-4117-ab1c-ddf9646a0ac6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:17:02.792233 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bdb6f3-bdc2-4117-ab1c-ddf9646a0ac6 service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:17:02.801197 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c28a57-cada-4ad5-94b5-d7ef9493e429 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb857f32bbc04d82bfa681bd5659765f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:17:02.802286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c28a57-cada-4ad5-94b5-d7ef9493e429 service placement] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:17:02.802722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c28a57-cada-4ad5-94b5-d7ef9493e429 service placement] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:17:02.803354 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c28a57-cada-4ad5-94b5-d7ef9493e429 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:17:02.812634 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561a10d-5758-4a3b-99eb-bbe68266df8b service neutron] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:17:02.821778 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-42c28a57-cada-4ad5-94b5-d7ef9493e429 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:17:02.828612 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561a10d-5758-4a3b-99eb-bbe68266df8b service neutron] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:17:02.843878 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c412b-8d99-4139-9fee-a6256ff0d092 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb857f32bbc04d82bfa681bd5659765f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:17:02.845892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c412b-8d99-4139-9fee-a6256ff0d092 service placement] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:17:02.846264 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c412b-8d99-4139-9fee-a6256ff0d092 service placement] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:17:02.846620 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c412b-8d99-4139-9fee-a6256ff0d092 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:17:02.847803 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c28a57-cada-4ad5-94b5-d7ef9493e429 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:17:02.849858 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c28a57-cada-4ad5-94b5-d7ef9493e429 service placement] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:17:02.849982 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-886c412b-8d99-4139-9fee-a6256ff0d092 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:17:02.867400 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886c412b-8d99-4139-9fee-a6256ff0d092 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:17:02.869300 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886c412b-8d99-4139-9fee-a6256ff0d092 service placement] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:17:02.871462 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1158/2317] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:17:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:17:02.883160 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1160/2318] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:17:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:17:06.898064 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c28a57-cada-4ad5-94b5-d7ef9493e429 service placement] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:17:06.899692 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-42c28a57-cada-4ad5-94b5-d7ef9493e429 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:17:06.914549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c28a57-cada-4ad5-94b5-d7ef9493e429 service placement] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:17:06.930031 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1709988-88e1-460d-9d19-1d871c22f6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:17:06.931251 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1709988-88e1-460d-9d19-1d871c22f6d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:17:06.931858 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1709988-88e1-460d-9d19-1d871c22f6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:17:06.932333 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1709988-88e1-460d-9d19-1d871c22f6d5 admin admin] PATH_INFO: `/v3/users/58016f5cae3d45eab6f9caf161e65c4c` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:17:06.933796 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1709988-88e1-460d-9d19-1d871c22f6d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58016f5cae3d45eab6f9caf161e65c4c)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:17:06.936254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1709988-88e1-460d-9d19-1d871c22f6d5 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:17:06.994827 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-b1709988-88e1-460d-9d19-1d871c22f6d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58016f5cae3d45eab6f9caf161e65c4c'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:07.016836 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-b1709988-88e1-460d-9d19-1d871c22f6d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58016f5cae3d45eab6f9caf161e65c4c'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:07.017539 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-b1709988-88e1-460d-9d19-1d871c22f6d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58016f5cae3d45eab6f9caf161e65c4c'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:07.018516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-b1709988-88e1-460d-9d19-1d871c22f6d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58016f5cae3d45eab6f9caf161e65c4c'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:07.028904 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-b1709988-88e1-460d-9d19-1d871c22f6d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58016f5cae3d45eab6f9caf161e65c4c'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:07.050496 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1159/2319] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:17:06 2022] DELETE /identity/v3/users/58016f5cae3d45eab6f9caf161e65c4c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:17:07.090146 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c412b-8d99-4139-9fee-a6256ff0d092 service placement] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:17:07.091708 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-886c412b-8d99-4139-9fee-a6256ff0d092 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:17:07.105573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c412b-8d99-4139-9fee-a6256ff0d092 service placement] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:17:07.176062 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e0e10-26af-4f35-8793-6f2b70f9ddba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:17:07.177013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e0e10-26af-4f35-8793-6f2b70f9ddba admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:17:07.177301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e0e10-26af-4f35-8793-6f2b70f9ddba admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:17:07.177897 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e0e10-26af-4f35-8793-6f2b70f9ddba admin admin] PATH_INFO: `/v3/users/dd610b017e7d4bd786db6bcd08e60722` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:17:07.193373 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e0e10-26af-4f35-8793-6f2b70f9ddba admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd610b017e7d4bd786db6bcd08e60722)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:17:07.195561 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e0e10-26af-4f35-8793-6f2b70f9ddba admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:17:07.257299 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-8c2e0e10-26af-4f35-8793-6f2b70f9ddba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd610b017e7d4bd786db6bcd08e60722'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:07.274175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1709988-88e1-460d-9d19-1d871c22f6d5 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:17:07.275718 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-b1709988-88e1-460d-9d19-1d871c22f6d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:17:07.282848 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-8c2e0e10-26af-4f35-8793-6f2b70f9ddba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd610b017e7d4bd786db6bcd08e60722'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:07.283597 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-8c2e0e10-26af-4f35-8793-6f2b70f9ddba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd610b017e7d4bd786db6bcd08e60722'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:07.284208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-8c2e0e10-26af-4f35-8793-6f2b70f9ddba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd610b017e7d4bd786db6bcd08e60722'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:07.291103 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1709988-88e1-460d-9d19-1d871c22f6d5 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:17:07.295860 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-8c2e0e10-26af-4f35-8793-6f2b70f9ddba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd610b017e7d4bd786db6bcd08e60722'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:07.296940 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc830a38-1b6c-4e61-a18a-42d01edcc4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:17:07.297901 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc830a38-1b6c-4e61-a18a-42d01edcc4d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:17:07.298257 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc830a38-1b6c-4e61-a18a-42d01edcc4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:17:07.298611 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc830a38-1b6c-4e61-a18a-42d01edcc4d2 admin admin] PATH_INFO: `/v3/projects/779e3cc39cf14719ba34b3d12cac5a01` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:17:07.299993 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc830a38-1b6c-4e61-a18a-42d01edcc4d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=779e3cc39cf14719ba34b3d12cac5a01)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:17:07.302205 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc830a38-1b6c-4e61-a18a-42d01edcc4d2 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:17:07.314307 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1161/2320] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:17:07 2022] DELETE /identity/v3/users/dd610b017e7d4bd786db6bcd08e60722 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:17:07.360996 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:17:07.366006 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-dc830a38-1b6c-4e61-a18a-42d01edcc4d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '779e3cc39cf14719ba34b3d12cac5a01', 'request_id': 'req-dc830a38-1b6c-4e61-a18a-42d01edcc4d2'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:07.372700 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-dc830a38-1b6c-4e61-a18a-42d01edcc4d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '779e3cc39cf14719ba34b3d12cac5a01', 'request_id': 'req-dc830a38-1b6c-4e61-a18a-42d01edcc4d2'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:07.380861 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-dc830a38-1b6c-4e61-a18a-42d01edcc4d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '779e3cc39cf14719ba34b3d12cac5a01', 'request_id': 'req-dc830a38-1b6c-4e61-a18a-42d01edcc4d2'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:07.393790 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-dc830a38-1b6c-4e61-a18a-42d01edcc4d2 admin admin] The token cache is being invalidate because project 779e3cc39cf14719ba34b3d12cac5a01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:17:07.394080 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-dc830a38-1b6c-4e61-a18a-42d01edcc4d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:07.394744 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-dc830a38-1b6c-4e61-a18a-42d01edcc4d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:07.396313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1160/2321] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:17:07 2022] DELETE /identity/v3/projects/779e3cc39cf14719ba34b3d12cac5a01 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:17:07.400662 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e0e10-26af-4f35-8793-6f2b70f9ddba admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:17:07.402318 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-8c2e0e10-26af-4f35-8793-6f2b70f9ddba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:17:07.424473 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:17:07.472650 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e0e10-26af-4f35-8793-6f2b70f9ddba admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:17:07.510003 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc830a38-1b6c-4e61-a18a-42d01edcc4d2 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:17:07.511617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-dc830a38-1b6c-4e61-a18a-42d01edcc4d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:17:07.557907 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8bfa5-1aee-4aff-9d30-f364ee675618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:17:07.558846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8bfa5-1aee-4aff-9d30-f364ee675618 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:17:07.559154 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8bfa5-1aee-4aff-9d30-f364ee675618 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:17:07.559462 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8bfa5-1aee-4aff-9d30-f364ee675618 admin admin] PATH_INFO: `/v3/users/18e5f9018c0143c991414b649a4c1260` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:17:07.573913 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f8bfa5-1aee-4aff-9d30-f364ee675618 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18e5f9018c0143c991414b649a4c1260)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:17:07.575932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f8bfa5-1aee-4aff-9d30-f364ee675618 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:17:07.588643 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc830a38-1b6c-4e61-a18a-42d01edcc4d2 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:17:07.593870 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804bd891-5ecd-4388-97aa-35761f3a2651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:17:07.594741 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bd891-5ecd-4388-97aa-35761f3a2651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:17:07.595021 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bd891-5ecd-4388-97aa-35761f3a2651 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:17:07.595334 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bd891-5ecd-4388-97aa-35761f3a2651 admin admin] PATH_INFO: `/v3/projects/2c8531e2e07a47eaae4c66d0b82df2d9` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:17:07.596596 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804bd891-5ecd-4388-97aa-35761f3a2651 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c8531e2e07a47eaae4c66d0b82df2d9)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:17:07.598660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804bd891-5ecd-4388-97aa-35761f3a2651 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:17:07.625793 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-74f8bfa5-1aee-4aff-9d30-f364ee675618 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18e5f9018c0143c991414b649a4c1260'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:07.639066 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-74f8bfa5-1aee-4aff-9d30-f364ee675618 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18e5f9018c0143c991414b649a4c1260'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:07.639792 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-74f8bfa5-1aee-4aff-9d30-f364ee675618 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18e5f9018c0143c991414b649a4c1260'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:07.640423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-74f8bfa5-1aee-4aff-9d30-f364ee675618 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18e5f9018c0143c991414b649a4c1260'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:07.647540 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-74f8bfa5-1aee-4aff-9d30-f364ee675618 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18e5f9018c0143c991414b649a4c1260'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:07.650072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-804bd891-5ecd-4388-97aa-35761f3a2651 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c8531e2e07a47eaae4c66d0b82df2d9', 'request_id': 'req-804bd891-5ecd-4388-97aa-35761f3a2651'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:07.656124 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-804bd891-5ecd-4388-97aa-35761f3a2651 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c8531e2e07a47eaae4c66d0b82df2d9', 'request_id': 'req-804bd891-5ecd-4388-97aa-35761f3a2651'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:07.661135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1162/2322] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:17:07 2022] DELETE /identity/v3/users/18e5f9018c0143c991414b649a4c1260 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:17:07.662677 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-804bd891-5ecd-4388-97aa-35761f3a2651 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c8531e2e07a47eaae4c66d0b82df2d9', 'request_id': 'req-804bd891-5ecd-4388-97aa-35761f3a2651'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:07.676007 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-804bd891-5ecd-4388-97aa-35761f3a2651 admin admin] The token cache is being invalidate because project 2c8531e2e07a47eaae4c66d0b82df2d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:17:07.676310 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-804bd891-5ecd-4388-97aa-35761f3a2651 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:07.676951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-804bd891-5ecd-4388-97aa-35761f3a2651 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:07.678881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1161/2323] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:17:07 2022] DELETE /identity/v3/projects/2c8531e2e07a47eaae4c66d0b82df2d9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:17:07.682676 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8bfa5-1aee-4aff-9d30-f364ee675618 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:17:07.684180 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-74f8bfa5-1aee-4aff-9d30-f364ee675618 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:17:07.725453 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:17:07.756741 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8bfa5-1aee-4aff-9d30-f364ee675618 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:17:07.761547 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202e927-928e-4d86-9d5c-ee94b1bccade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:17:07.762422 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202e927-928e-4d86-9d5c-ee94b1bccade admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:17:07.762707 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202e927-928e-4d86-9d5c-ee94b1bccade admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:17:07.762990 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202e927-928e-4d86-9d5c-ee94b1bccade admin admin] PATH_INFO: `/v3/users/513b043962cf48c695c7c724bd08395f` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:17:07.776684 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202e927-928e-4d86-9d5c-ee94b1bccade admin admin] RBAC: Authorizing `identity:delete_user(user_id=513b043962cf48c695c7c724bd08395f)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:17:07.778636 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202e927-928e-4d86-9d5c-ee94b1bccade admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:17:07.842155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-8202e927-928e-4d86-9d5c-ee94b1bccade admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '513b043962cf48c695c7c724bd08395f'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:07.854839 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-8202e927-928e-4d86-9d5c-ee94b1bccade admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '513b043962cf48c695c7c724bd08395f'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:07.855713 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-8202e927-928e-4d86-9d5c-ee94b1bccade admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '513b043962cf48c695c7c724bd08395f'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:07.856424 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-8202e927-928e-4d86-9d5c-ee94b1bccade admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '513b043962cf48c695c7c724bd08395f'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:07.859638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804bd891-5ecd-4388-97aa-35761f3a2651 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:17:07.861137 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-804bd891-5ecd-4388-97aa-35761f3a2651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:17:07.866784 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-8202e927-928e-4d86-9d5c-ee94b1bccade admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '513b043962cf48c695c7c724bd08395f'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:07.882838 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1163/2324] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:17:07 2022] DELETE /identity/v3/users/513b043962cf48c695c7c724bd08395f => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:17:07.957140 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804bd891-5ecd-4388-97aa-35761f3a2651 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:17:07.966385 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:17:08.020721 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a8171-c885-40ab-ab5e-2fe30a3d74ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:17:08.021604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a8171-c885-40ab-ab5e-2fe30a3d74ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:17:08.021914 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a8171-c885-40ab-ab5e-2fe30a3d74ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:17:08.022187 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a8171-c885-40ab-ab5e-2fe30a3d74ba admin admin] PATH_INFO: `/v3/projects/e74f19cd54e84ccba8aa971edec4fb5e` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:17:08.033650 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a8171-c885-40ab-ab5e-2fe30a3d74ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=e74f19cd54e84ccba8aa971edec4fb5e)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:17:08.035620 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a8171-c885-40ab-ab5e-2fe30a3d74ba admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:17:08.069703 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202e927-928e-4d86-9d5c-ee94b1bccade admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:17:08.071314 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-8202e927-928e-4d86-9d5c-ee94b1bccade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:17:08.085161 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-4e8a8171-c885-40ab-ab5e-2fe30a3d74ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e74f19cd54e84ccba8aa971edec4fb5e', 'request_id': 'req-4e8a8171-c885-40ab-ab5e-2fe30a3d74ba'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:08.086708 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202e927-928e-4d86-9d5c-ee94b1bccade admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:17:08.091838 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-4e8a8171-c885-40ab-ab5e-2fe30a3d74ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e74f19cd54e84ccba8aa971edec4fb5e', 'request_id': 'req-4e8a8171-c885-40ab-ab5e-2fe30a3d74ba'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:08.099611 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-4e8a8171-c885-40ab-ab5e-2fe30a3d74ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e74f19cd54e84ccba8aa971edec4fb5e', 'request_id': 'req-4e8a8171-c885-40ab-ab5e-2fe30a3d74ba'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:08.115274 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-4e8a8171-c885-40ab-ab5e-2fe30a3d74ba admin admin] The token cache is being invalidate because project e74f19cd54e84ccba8aa971edec4fb5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:17:08.115567 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-4e8a8171-c885-40ab-ab5e-2fe30a3d74ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:08.116273 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-4e8a8171-c885-40ab-ab5e-2fe30a3d74ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:08.117891 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1162/2325] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:17:07 2022] DELETE /identity/v3/projects/e74f19cd54e84ccba8aa971edec4fb5e => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:17:08.151319 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020536d-be07-4d9b-9a67-2c7b75ece5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:17:08.152129 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020536d-be07-4d9b-9a67-2c7b75ece5bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:17:08.152564 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020536d-be07-4d9b-9a67-2c7b75ece5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:17:08.152989 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020536d-be07-4d9b-9a67-2c7b75ece5bb admin admin] PATH_INFO: `/v3/projects/1cc85169a9644cd28b5cd790efac0680` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:17:08.165483 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1020536d-be07-4d9b-9a67-2c7b75ece5bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cc85169a9644cd28b5cd790efac0680)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:17:08.167521 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1020536d-be07-4d9b-9a67-2c7b75ece5bb admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:17:08.210541 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:17:08.222369 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1020536d-be07-4d9b-9a67-2c7b75ece5bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cc85169a9644cd28b5cd790efac0680', 'request_id': 'req-1020536d-be07-4d9b-9a67-2c7b75ece5bb'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:08.228711 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1020536d-be07-4d9b-9a67-2c7b75ece5bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cc85169a9644cd28b5cd790efac0680', 'request_id': 'req-1020536d-be07-4d9b-9a67-2c7b75ece5bb'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:08.237509 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1020536d-be07-4d9b-9a67-2c7b75ece5bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cc85169a9644cd28b5cd790efac0680', 'request_id': 'req-1020536d-be07-4d9b-9a67-2c7b75ece5bb'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:08.253194 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1020536d-be07-4d9b-9a67-2c7b75ece5bb admin admin] The token cache is being invalidate because project 1cc85169a9644cd28b5cd790efac0680 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:17:08.253486 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1020536d-be07-4d9b-9a67-2c7b75ece5bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:08.254262 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1020536d-be07-4d9b-9a67-2c7b75ece5bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:17:08.255955 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1164/2326] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:17:08 2022] DELETE /identity/v3/projects/1cc85169a9644cd28b5cd790efac0680 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:17:18.112438 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=50,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:17:18.250348 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:DELETE=5,keystone:SELECT=11,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:18:53.820767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a8171-c885-40ab-ab5e-2fe30a3d74ba admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:18:53.822346 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-4e8a8171-c885-40ab-ab5e-2fe30a3d74ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:18:53.880991 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a8171-c885-40ab-ab5e-2fe30a3d74ba admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:18:53.930289 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f76cf4-8905-4564-a930-4e523334dbae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:18:53.930961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f76cf4-8905-4564-a930-4e523334dbae service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:18:53.931275 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f76cf4-8905-4564-a930-4e523334dbae service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:18:53.931554 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f76cf4-8905-4564-a930-4e523334dbae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:18:53.933367 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-49f76cf4-8905-4564-a930-4e523334dbae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:18:54.009989 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f76cf4-8905-4564-a930-4e523334dbae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:18:54.011442 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f76cf4-8905-4564-a930-4e523334dbae service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:18:54.020765 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1163/2327] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:18:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:19:04.016533 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=73 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:19:06.455820 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020536d-be07-4d9b-9a67-2c7b75ece5bb admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:19:06.457450 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-1020536d-be07-4d9b-9a67-2c7b75ece5bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:19:06.538896 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020536d-be07-4d9b-9a67-2c7b75ece5bb admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:19:06.543625 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9090fd-2d01-4f20-8457-f6b7b0df0f3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:19:06.544272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9090fd-2d01-4f20-8457-f6b7b0df0f3e service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:19:06.544555 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9090fd-2d01-4f20-8457-f6b7b0df0f3e service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:19:06.544840 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9090fd-2d01-4f20-8457-f6b7b0df0f3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:19:06.547150 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-3f9090fd-2d01-4f20-8457-f6b7b0df0f3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:19:06.557936 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9090fd-2d01-4f20-8457-f6b7b0df0f3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:19:06.559416 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9090fd-2d01-4f20-8457-f6b7b0df0f3e service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:19:06.568122 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1165/2328] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:19:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:19:16.564271 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=30 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:20:54.945861 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f76cf4-8905-4564-a930-4e523334dbae service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:20:54.963706 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f76cf4-8905-4564-a930-4e523334dbae service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:20:54.968433 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915a3ee-961c-4266-baa4-00fbabf98943 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:20:54.969153 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915a3ee-961c-4266-baa4-00fbabf98943 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:20:54.969438 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915a3ee-961c-4266-baa4-00fbabf98943 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:20:54.969713 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915a3ee-961c-4266-baa4-00fbabf98943 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:20:54.971772 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-1915a3ee-961c-4266-baa4-00fbabf98943 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:20:55.052188 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915a3ee-961c-4266-baa4-00fbabf98943 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:20:55.053666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915a3ee-961c-4266-baa4-00fbabf98943 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:20:55.075826 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1164/2329] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:20:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:20:57.468675 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9090fd-2d01-4f20-8457-f6b7b0df0f3e service neutron] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:20:57.470321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-3f9090fd-2d01-4f20-8457-f6b7b0df0f3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:20:57.502439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9090fd-2d01-4f20-8457-f6b7b0df0f3e service neutron] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:20:57.509800 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ddeb2e-6e1a-4482-a087-42f9b5ef7705 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:20:57.510995 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ddeb2e-6e1a-4482-a087-42f9b5ef7705 service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:20:57.511617 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ddeb2e-6e1a-4482-a087-42f9b5ef7705 service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:20:57.512198 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ddeb2e-6e1a-4482-a087-42f9b5ef7705 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:20:57.515547 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-91ddeb2e-6e1a-4482-a087-42f9b5ef7705 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:20:57.530248 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ddeb2e-6e1a-4482-a087-42f9b5ef7705 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:20:57.531815 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ddeb2e-6e1a-4482-a087-42f9b5ef7705 service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:20:57.544113 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1166/2330] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:20:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:21:05.072984 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=43 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:21:07.539373 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=8 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:21:21.726708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915a3ee-961c-4266-baa4-00fbabf98943 service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:21:21.728495 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-1915a3ee-961c-4266-baa4-00fbabf98943 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:21:21.807993 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915a3ee-961c-4266-baa4-00fbabf98943 service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:21:21.814228 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132700c3-a8c0-4507-9f4d-5342ddfea6c6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb857f32bbc04d82bfa681bd5659765f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:21:21.814921 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132700c3-a8c0-4507-9f4d-5342ddfea6c6 service placement] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:21:21.815283 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132700c3-a8c0-4507-9f4d-5342ddfea6c6 service placement] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:21:21.815563 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132700c3-a8c0-4507-9f4d-5342ddfea6c6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:21:21.817440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-132700c3-a8c0-4507-9f4d-5342ddfea6c6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:21:21.829731 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132700c3-a8c0-4507-9f4d-5342ddfea6c6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:21:21.831510 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132700c3-a8c0-4507-9f4d-5342ddfea6c6 service placement] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:21:21.842175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1165/2331] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:21:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:21:31.837750 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=32 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:21:53.042366 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ddeb2e-6e1a-4482-a087-42f9b5ef7705 service nova] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:21:53.055365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ddeb2e-6e1a-4482-a087-42f9b5ef7705 service nova] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:21:53.059082 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132700c3-a8c0-4507-9f4d-5342ddfea6c6 service placement] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:21:53.060867 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-132700c3-a8c0-4507-9f4d-5342ddfea6c6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:21:53.063459 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c72cda-8cf3-4594-880d-bd143d8c5631 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:21:53.064170 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c72cda-8cf3-4594-880d-bd143d8c5631 service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:21:53.064485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c72cda-8cf3-4594-880d-bd143d8c5631 service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:21:53.064840 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c72cda-8cf3-4594-880d-bd143d8c5631 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:21:53.067216 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-99c72cda-8cf3-4594-880d-bd143d8c5631 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:21:53.079426 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132700c3-a8c0-4507-9f4d-5342ddfea6c6 service placement] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:21:53.085458 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a6bc1-80cd-4e82-9b96-df32db91281e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:21:53.086123 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a6bc1-80cd-4e82-9b96-df32db91281e service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:21:53.086408 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a6bc1-80cd-4e82-9b96-df32db91281e service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:21:53.086689 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a6bc1-80cd-4e82-9b96-df32db91281e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:21:53.088581 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-ca6a6bc1-80cd-4e82-9b96-df32db91281e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:21:53.129919 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c72cda-8cf3-4594-880d-bd143d8c5631 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:21:53.131993 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c72cda-8cf3-4594-880d-bd143d8c5631 service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:21:53.141620 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1167/2332] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:21:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:21:53.160106 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6a6bc1-80cd-4e82-9b96-df32db91281e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:21:53.161679 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6a6bc1-80cd-4e82-9b96-df32db91281e service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:21:53.181344 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1166/2333] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:21:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:21:53.234848 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c72cda-8cf3-4594-880d-bd143d8c5631 service nova] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:21:53.245137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c72cda-8cf3-4594-880d-bd143d8c5631 service nova] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:21:53.249059 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863866ae-21ad-4810-b4de-95574cd68902 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:21:53.249715 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863866ae-21ad-4810-b4de-95574cd68902 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:21:53.249998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863866ae-21ad-4810-b4de-95574cd68902 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:21:53.250273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863866ae-21ad-4810-b4de-95574cd68902 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:21:53.252822 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-863866ae-21ad-4810-b4de-95574cd68902 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:21:53.267396 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863866ae-21ad-4810-b4de-95574cd68902 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:21:53.268840 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863866ae-21ad-4810-b4de-95574cd68902 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:21:53.280252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1168/2334] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:21:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:21:53.320276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a6bc1-80cd-4e82-9b96-df32db91281e service nova] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:21:53.331821 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a6bc1-80cd-4e82-9b96-df32db91281e service nova] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:21:53.338984 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c15ab0-5aec-4583-a2f4-da7ad6289041 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:21:53.339657 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c15ab0-5aec-4583-a2f4-da7ad6289041 service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:21:53.339971 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c15ab0-5aec-4583-a2f4-da7ad6289041 service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:21:53.340238 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c15ab0-5aec-4583-a2f4-da7ad6289041 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:21:53.342303 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-c5c15ab0-5aec-4583-a2f4-da7ad6289041 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:21:53.430501 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c15ab0-5aec-4583-a2f4-da7ad6289041 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:21:53.432219 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c15ab0-5aec-4583-a2f4-da7ad6289041 service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:21:53.451472 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1167/2335] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:21:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:21:53.502830 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863866ae-21ad-4810-b4de-95574cd68902 service neutron] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:21:53.513883 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863866ae-21ad-4810-b4de-95574cd68902 service neutron] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:21:53.520059 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87448d-a26b-424c-82b2-6f088f23ae18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:21:53.520694 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87448d-a26b-424c-82b2-6f088f23ae18 service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:21:53.520999 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87448d-a26b-424c-82b2-6f088f23ae18 service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:21:53.521249 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87448d-a26b-424c-82b2-6f088f23ae18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:21:53.523174 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-2b87448d-a26b-424c-82b2-6f088f23ae18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:21:53.596944 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b87448d-a26b-424c-82b2-6f088f23ae18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:21:53.599184 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b87448d-a26b-424c-82b2-6f088f23ae18 service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:21:53.626769 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1169/2336] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:21:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:22:03.448883 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=77 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:03.624513 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=70 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:54.776229 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c15ab0-5aec-4583-a2f4-da7ad6289041 service nova] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:54.779104 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-c5c15ab0-5aec-4583-a2f4-da7ad6289041 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:54.805488 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c15ab0-5aec-4583-a2f4-da7ad6289041 service nova] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:54.809215 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc072f-61c7-49b4-89f1-aef417d96e46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:54.809871 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc072f-61c7-49b4-89f1-aef417d96e46 service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:54.810161 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc072f-61c7-49b4-89f1-aef417d96e46 service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:54.810439 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc072f-61c7-49b4-89f1-aef417d96e46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:54.812217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-46cc072f-61c7-49b4-89f1-aef417d96e46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:54.825376 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application [None req-46cc072f-61c7-49b4-89f1-aef417d96e46 service cinder] Could not find project: 779e3cc39cf14719ba34b3d12cac5a01.: keystone.exception.ProjectNotFound: Could not find project: 779e3cc39cf14719ba34b3d12cac5a01. Jul 20 15:22:54.825376 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:22:54.825376 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:22:54.825376 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:22:54.825376 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:22:54.825376 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:22:54.825376 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:22:54.825376 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:22:54.825376 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:22:54.825376 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:22:54.825376 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:22:54.825376 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:22:54.825376 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Jul 20 15:22:54.825376 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Jul 20 15:22:54.825376 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Jul 20 15:22:54.826619 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Jul 20 15:22:54.826619 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Jul 20 15:22:54.826619 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Jul 20 15:22:54.826619 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:22:54.826619 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:22:54.826619 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Jul 20 15:22:54.826619 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Jul 20 15:22:54.826619 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:22:54.826619 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:22:54.826619 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:22:54.826619 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:22:54.826619 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:22:54.826619 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 15:22:54.826619 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:22:54.826619 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:22:54.826619 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:22:54.826619 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:22:54.828233 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:22:54.828233 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:22:54.828233 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:22:54.828233 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:22:54.828233 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Jul 20 15:22:54.828233 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Jul 20 15:22:54.828233 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Jul 20 15:22:54.828233 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application self._validate_token_resources() Jul 20 15:22:54.828233 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Jul 20 15:22:54.828233 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Jul 20 15:22:54.828233 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Jul 20 15:22:54.828233 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Jul 20 15:22:54.828233 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:22:54.828233 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:22:54.828233 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:22:54.828233 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:22:54.828233 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:22:54.829473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:22:54.829473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:22:54.829473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application with Lock( Jul 20 15:22:54.829473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:22:54.829473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:22:54.829473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:22:54.829473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:22:54.829473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:22:54.829473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:22:54.829473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:22:54.829473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:22:54.829473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:22:54.829473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:22:54.829473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:22:54.829473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:22:54.829473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:22:54.829473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:22:54.830592 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 779e3cc39cf14719ba34b3d12cac5a01. Jul 20 15:22:54.830592 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: ERROR keystone.server.flask.application Jul 20 15:22:54.830592 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1168/2337] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:22:54 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 52 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Jul 20 15:23:04.816148 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=8 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:10.393239 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87448d-a26b-424c-82b2-6f088f23ae18 service nova] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:10.403722 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87448d-a26b-424c-82b2-6f088f23ae18 service nova] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:10.413843 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e111f-cab2-4024-876f-1a2022cfba20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:10.414579 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e111f-cab2-4024-876f-1a2022cfba20 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:10.414871 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e111f-cab2-4024-876f-1a2022cfba20 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:10.415165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e111f-cab2-4024-876f-1a2022cfba20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:10.423621 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e111f-cab2-4024-876f-1a2022cfba20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:10.425026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e111f-cab2-4024-876f-1a2022cfba20 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:10.433415 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1170/2338] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:24:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:24:20.429895 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=6 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:28.778305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc072f-61c7-49b4-89f1-aef417d96e46 service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:28.792866 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc072f-61c7-49b4-89f1-aef417d96e46 service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:28.799441 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927114ba-7221-4dc0-b897-9488b3ab5c2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:28.800096 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927114ba-7221-4dc0-b897-9488b3ab5c2b service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:28.800381 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927114ba-7221-4dc0-b897-9488b3ab5c2b service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:28.800657 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927114ba-7221-4dc0-b897-9488b3ab5c2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:28.812477 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927114ba-7221-4dc0-b897-9488b3ab5c2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:28.814104 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927114ba-7221-4dc0-b897-9488b3ab5c2b service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:28.843982 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1169/2339] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:24:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:24:38.841078 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=13 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:55.304868 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114af428-d7f0-40b2-a2e5-04f2b8ea2f01 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:55.305342 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114af428-d7f0-40b2-a2e5-04f2b8ea2f01 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:55.305533 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114af428-d7f0-40b2-a2e5-04f2b8ea2f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:55.334770 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-114af428-d7f0-40b2-a2e5-04f2b8ea2f01 None None] MFA Rules not processed for user `5f51124af1e846fb89552078c81afdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:55.336010 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-114af428-d7f0-40b2-a2e5-04f2b8ea2f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:55.343096 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1171/2340] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:24:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:24:55.515543 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbee29-28c3-4882-8d12-042610bc4157 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:55.516043 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbee29-28c3-4882-8d12-042610bc4157 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:55.516157 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbee29-28c3-4882-8d12-042610bc4157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:55.547936 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-1bcbee29-28c3-4882-8d12-042610bc4157 None None] MFA Rules not processed for user `e3574b5b8f3241b192b0b810122d4769`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:55.549590 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-1bcbee29-28c3-4882-8d12-042610bc4157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:55.561492 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1170/2341] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:24:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:24:57.395891 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7e877-0e5f-4581-9a8e-abdf6d64264d None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:57.396454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7e877-0e5f-4581-9a8e-abdf6d64264d None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:57.396454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7e877-0e5f-4581-9a8e-abdf6d64264d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:57.425349 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-dbb7e877-0e5f-4581-9a8e-abdf6d64264d None None] MFA Rules not processed for user `bc2f1a0d431341d6a5d1a8c5de131ece`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:57.426585 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-dbb7e877-0e5f-4581-9a8e-abdf6d64264d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:57.434111 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1172/2342] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:24:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:24:57.862205 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcbee29-28c3-4882-8d12-042610bc4157 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:57.873217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcbee29-28c3-4882-8d12-042610bc4157 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:57.877146 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a915a-c0c8-4d6f-967d-d6ea8f75fd11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:57.877811 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a915a-c0c8-4d6f-967d-d6ea8f75fd11 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:57.878103 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a915a-c0c8-4d6f-967d-d6ea8f75fd11 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:57.878387 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a915a-c0c8-4d6f-967d-d6ea8f75fd11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:57.880302 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-658a915a-c0c8-4d6f-967d-d6ea8f75fd11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:57.903916 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a915a-c0c8-4d6f-967d-d6ea8f75fd11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:57.905417 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a915a-c0c8-4d6f-967d-d6ea8f75fd11 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:57.914467 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1171/2343] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:24:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:24:58.396005 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7e877-0e5f-4581-9a8e-abdf6d64264d None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:58.397922 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-dbb7e877-0e5f-4581-9a8e-abdf6d64264d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:58.416947 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7e877-0e5f-4581-9a8e-abdf6d64264d None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:58.423085 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c6b5d-3391-4393-83ae-d33f4e62f5ef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb857f32bbc04d82bfa681bd5659765f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:58.423935 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c6b5d-3391-4393-83ae-d33f4e62f5ef service placement] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:58.424303 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c6b5d-3391-4393-83ae-d33f4e62f5ef service placement] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:58.424784 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c6b5d-3391-4393-83ae-d33f4e62f5ef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:58.427500 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-612c6b5d-3391-4393-83ae-d33f4e62f5ef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:58.440931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612c6b5d-3391-4393-83ae-d33f4e62f5ef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:58.442669 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612c6b5d-3391-4393-83ae-d33f4e62f5ef service placement] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:58.457318 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1173/2344] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:24:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:25:02.417381 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a915a-c0c8-4d6f-967d-d6ea8f75fd11 service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:02.441278 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a915a-c0c8-4d6f-967d-d6ea8f75fd11 service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:02.450947 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c3947-7013-4ed7-bf60-270500a94821 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:02.451958 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c3947-7013-4ed7-bf60-270500a94821 service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:02.452447 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c3947-7013-4ed7-bf60-270500a94821 service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:02.452926 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c3947-7013-4ed7-bf60-270500a94821 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:02.455138 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-ee2c3947-7013-4ed7-bf60-270500a94821 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:02.475373 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c3947-7013-4ed7-bf60-270500a94821 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:02.478771 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c3947-7013-4ed7-bf60-270500a94821 service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:02.499118 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1172/2345] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:25:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:25:04.883575 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6d95b-71a0-4757-981a-50c0a9eaa69a None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:04.884588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6d95b-71a0-4757-981a-50c0a9eaa69a None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:04.884968 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6d95b-71a0-4757-981a-50c0a9eaa69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:04.918809 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-8af6d95b-71a0-4757-981a-50c0a9eaa69a None None] MFA Rules not processed for user `ee5a234904e64eb9b2faad85909d0507`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:04.920396 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-8af6d95b-71a0-4757-981a-50c0a9eaa69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:04.928411 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1174/2346] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:25:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:25:10.903175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c3947-7013-4ed7-bf60-270500a94821 service nova] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:10.916689 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c3947-7013-4ed7-bf60-270500a94821 service nova] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:10.922731 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284f753-9b11-4f60-811a-3fcf29ce20b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:10.923485 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284f753-9b11-4f60-811a-3fcf29ce20b1 service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:10.923775 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284f753-9b11-4f60-811a-3fcf29ce20b1 service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:10.924060 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284f753-9b11-4f60-811a-3fcf29ce20b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:10.925970 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-a284f753-9b11-4f60-811a-3fcf29ce20b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:10.939623 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a284f753-9b11-4f60-811a-3fcf29ce20b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:10.941643 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a284f753-9b11-4f60-811a-3fcf29ce20b1 service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:10.953800 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1173/2347] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:25:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:25:14.915673 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=24 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:20.948956 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=29 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:24.600002 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af6d95b-71a0-4757-981a-50c0a9eaa69a None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:24.601658 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-8af6d95b-71a0-4757-981a-50c0a9eaa69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:24.616270 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af6d95b-71a0-4757-981a-50c0a9eaa69a None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:24.620529 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024f24d-be2d-447d-bf0b-fd6896326578 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:24.621565 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024f24d-be2d-447d-bf0b-fd6896326578 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:24.622073 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024f24d-be2d-447d-bf0b-fd6896326578 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:24.622590 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024f24d-be2d-447d-bf0b-fd6896326578 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:24.624856 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-d024f24d-be2d-447d-bf0b-fd6896326578 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:24.693440 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d024f24d-be2d-447d-bf0b-fd6896326578 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:24.694840 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d024f24d-be2d-447d-bf0b-fd6896326578 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:24.704791 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1175/2348] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:25:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:25:28.120425 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284f753-9b11-4f60-811a-3fcf29ce20b1 service nova] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:28.148738 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284f753-9b11-4f60-811a-3fcf29ce20b1 service nova] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:28.158113 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5121319-65bf-4817-b3af-22d414382b4d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb857f32bbc04d82bfa681bd5659765f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:28.159635 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5121319-65bf-4817-b3af-22d414382b4d service placement] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:28.160834 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5121319-65bf-4817-b3af-22d414382b4d service placement] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:28.161132 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5121319-65bf-4817-b3af-22d414382b4d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:28.163596 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e5121319-65bf-4817-b3af-22d414382b4d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:28.177390 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5121319-65bf-4817-b3af-22d414382b4d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:28.179916 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5121319-65bf-4817-b3af-22d414382b4d service placement] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:28.195608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1174/2349] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:25:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:25:28.229268 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024f24d-be2d-447d-bf0b-fd6896326578 service neutron] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:28.230827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-d024f24d-be2d-447d-bf0b-fd6896326578 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:28.244376 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024f24d-be2d-447d-bf0b-fd6896326578 service neutron] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:28.249560 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b739aa-ac06-448c-a663-03caf50275a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:28.250457 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b739aa-ac06-448c-a663-03caf50275a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:28.250736 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b739aa-ac06-448c-a663-03caf50275a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:28.251009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b739aa-ac06-448c-a663-03caf50275a7 admin admin] PATH_INFO: `/v3/users/938083ff74d84224be142a26d8e17ad6` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:28.252390 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b739aa-ac06-448c-a663-03caf50275a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=938083ff74d84224be142a26d8e17ad6)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:28.254522 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b739aa-ac06-448c-a663-03caf50275a7 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:28.314694 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-02b739aa-ac06-448c-a663-03caf50275a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '938083ff74d84224be142a26d8e17ad6'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:28.328512 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-02b739aa-ac06-448c-a663-03caf50275a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '938083ff74d84224be142a26d8e17ad6'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:28.329420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-02b739aa-ac06-448c-a663-03caf50275a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '938083ff74d84224be142a26d8e17ad6'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:28.330240 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-02b739aa-ac06-448c-a663-03caf50275a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '938083ff74d84224be142a26d8e17ad6'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:28.337683 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-02b739aa-ac06-448c-a663-03caf50275a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '938083ff74d84224be142a26d8e17ad6'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:28.353196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1176/2350] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:25:28 2022] DELETE /identity/v3/users/938083ff74d84224be142a26d8e17ad6 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:25:28.536287 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5121319-65bf-4817-b3af-22d414382b4d service placement] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:28.548631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5121319-65bf-4817-b3af-22d414382b4d service placement] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:28.609882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c286d8-2d18-4283-8dda-ded193ef8de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:28.611033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c286d8-2d18-4283-8dda-ded193ef8de0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:28.611375 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c286d8-2d18-4283-8dda-ded193ef8de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:28.611650 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c286d8-2d18-4283-8dda-ded193ef8de0 admin admin] PATH_INFO: `/v3/projects/5c6ce84a298f4daa938864ec78d7a996` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:28.613016 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c286d8-2d18-4283-8dda-ded193ef8de0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c6ce84a298f4daa938864ec78d7a996)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:28.615167 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c286d8-2d18-4283-8dda-ded193ef8de0 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:28.666974 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-81c286d8-2d18-4283-8dda-ded193ef8de0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c6ce84a298f4daa938864ec78d7a996', 'request_id': 'req-81c286d8-2d18-4283-8dda-ded193ef8de0'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:28.672990 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-81c286d8-2d18-4283-8dda-ded193ef8de0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c6ce84a298f4daa938864ec78d7a996', 'request_id': 'req-81c286d8-2d18-4283-8dda-ded193ef8de0'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:28.679516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-81c286d8-2d18-4283-8dda-ded193ef8de0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c6ce84a298f4daa938864ec78d7a996', 'request_id': 'req-81c286d8-2d18-4283-8dda-ded193ef8de0'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:28.695990 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-81c286d8-2d18-4283-8dda-ded193ef8de0 admin admin] The token cache is being invalidate because project 5c6ce84a298f4daa938864ec78d7a996 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:25:28.696285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-81c286d8-2d18-4283-8dda-ded193ef8de0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:28.696964 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-81c286d8-2d18-4283-8dda-ded193ef8de0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:28.698562 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1175/2351] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:25:28 2022] DELETE /identity/v3/projects/5c6ce84a298f4daa938864ec78d7a996 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:25:28.703611 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3fee4-d78c-4335-ac3d-9d7355e59e44 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:28.703904 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3fee4-d78c-4335-ac3d-9d7355e59e44 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:28.704184 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3fee4-d78c-4335-ac3d-9d7355e59e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:28.762665 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-c0b3fee4-d78c-4335-ac3d-9d7355e59e44 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:28.763923 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-c0b3fee4-d78c-4335-ac3d-9d7355e59e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:28.794874 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:28.824953 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1177/2352] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:25:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:25:28.833322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca845c-e382-4ab6-af6d-73454ea0314d None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:28.833607 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca845c-e382-4ab6-af6d-73454ea0314d None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:28.833932 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca845c-e382-4ab6-af6d-73454ea0314d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:28.852339 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-11ca845c-e382-4ab6-af6d-73454ea0314d None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:28.853545 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-11ca845c-e382-4ab6-af6d-73454ea0314d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:28.862148 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1176/2353] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:25:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:25:28.866567 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3fee4-d78c-4335-ac3d-9d7355e59e44 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:28.876914 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3fee4-d78c-4335-ac3d-9d7355e59e44 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:28.882815 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8aa68-3649-407b-b935-26f0c80c4cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:28.883550 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8aa68-3649-407b-b935-26f0c80c4cc4 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:28.883840 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8aa68-3649-407b-b935-26f0c80c4cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:28.884122 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8aa68-3649-407b-b935-26f0c80c4cc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:28.884862 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8aa68-3649-407b-b935-26f0c80c4cc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:25:28.885176 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8aa68-3649-407b-b935-26f0c80c4cc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:28.886811 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8aa68-3649-407b-b935-26f0c80c4cc4 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:28.899103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1178/2354] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 15:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:25:28.903542 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca845c-e382-4ab6-af6d-73454ea0314d None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:28.913115 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca845c-e382-4ab6-af6d-73454ea0314d None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:28.918417 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485e66b-2fc8-434a-9d9e-7c1536f0d0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:28.919037 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485e66b-2fc8-434a-9d9e-7c1536f0d0b9 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:28.919361 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485e66b-2fc8-434a-9d9e-7c1536f0d0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:28.919644 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485e66b-2fc8-434a-9d9e-7c1536f0d0b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:28.920356 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485e66b-2fc8-434a-9d9e-7c1536f0d0b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:28.922306 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485e66b-2fc8-434a-9d9e-7c1536f0d0b9 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:28.960072 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1177/2355] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:25:28 2022] POST /identity/v3/projects => generated 379 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:25:28.964303 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8aa68-3649-407b-b935-26f0c80c4cc4 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:28.974876 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8aa68-3649-407b-b935-26f0c80c4cc4 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:29.035457 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b83726-453c-44e3-8dfc-25686df8a0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:29.036125 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b83726-453c-44e3-8dfc-25686df8a0cc admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:29.036410 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b83726-453c-44e3-8dfc-25686df8a0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:29.036663 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b83726-453c-44e3-8dfc-25686df8a0cc admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:29.037371 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b83726-453c-44e3-8dfc-25686df8a0cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:29.039347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b83726-453c-44e3-8dfc-25686df8a0cc admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:29.059675 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1179/2356] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:25:28 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:25:29.063993 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485e66b-2fc8-434a-9d9e-7c1536f0d0b9 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:29.075996 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485e66b-2fc8-434a-9d9e-7c1536f0d0b9 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:29.081087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb2f0bb-3b35-4e68-93f1-81fcee0aeb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:29.081836 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2f0bb-3b35-4e68-93f1-81fcee0aeb2f admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:29.082127 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2f0bb-3b35-4e68-93f1-81fcee0aeb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:29.082515 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2f0bb-3b35-4e68-93f1-81fcee0aeb2f admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:29.083285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb2f0bb-3b35-4e68-93f1-81fcee0aeb2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:29.084909 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb2f0bb-3b35-4e68-93f1-81fcee0aeb2f admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:29.095171 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1178/2357] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:25:29.099361 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b83726-453c-44e3-8dfc-25686df8a0cc admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:29.109806 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b83726-453c-44e3-8dfc-25686df8a0cc admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:29.116194 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d9d71-31d6-4b45-aa4b-03f26506d633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:29.116875 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d9d71-31d6-4b45-aa4b-03f26506d633 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:29.117138 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d9d71-31d6-4b45-aa4b-03f26506d633 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:29.117406 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d9d71-31d6-4b45-aa4b-03f26506d633 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:29.118133 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d9d71-31d6-4b45-aa4b-03f26506d633 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:29.119784 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d9d71-31d6-4b45-aa4b-03f26506d633 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:29.130209 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1180/2358] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:25:29.134423 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb2f0bb-3b35-4e68-93f1-81fcee0aeb2f admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:29.144970 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb2f0bb-3b35-4e68-93f1-81fcee0aeb2f admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:29.150230 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e4ce0-a09b-4a55-9c8f-5448a3f2c83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:29.151132 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e4ce0-a09b-4a55-9c8f-5448a3f2c83b admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:29.151458 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e4ce0-a09b-4a55-9c8f-5448a3f2c83b admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:29.151726 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e4ce0-a09b-4a55-9c8f-5448a3f2c83b admin admin] PATH_INFO: `/v3/projects/eb8017f7f7ff42079a53548e60ed2d9d/users/ca06c3fab691475184f461c4954a7d7d/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:29.167014 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6e4ce0-a09b-4a55-9c8f-5448a3f2c83b admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb8017f7f7ff42079a53548e60ed2d9d, user_id=ca06c3fab691475184f461c4954a7d7d, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:29.169742 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6e4ce0-a09b-4a55-9c8f-5448a3f2c83b admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:29.184350 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1179/2359] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:25:29 2022] PUT /identity/v3/projects/eb8017f7f7ff42079a53548e60ed2d9d/users/ca06c3fab691475184f461c4954a7d7d/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:25:29.195784 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d9ed1-a9da-4e14-8fa5-eab2f4d9051f None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:29.196071 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d9ed1-a9da-4e14-8fa5-eab2f4d9051f None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:29.196353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d9ed1-a9da-4e14-8fa5-eab2f4d9051f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:29.224219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-098d9ed1-a9da-4e14-8fa5-eab2f4d9051f None None] MFA Rules not processed for user `ca06c3fab691475184f461c4954a7d7d`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:29.225443 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-098d9ed1-a9da-4e14-8fa5-eab2f4d9051f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:29.291295 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1181/2360] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:25:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:25:29.295822 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e4ce0-a09b-4a55-9c8f-5448a3f2c83b admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:29.301960 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:29.363578 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e4ce0-a09b-4a55-9c8f-5448a3f2c83b admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:29.368702 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b899dd-2a13-4d76-ad1b-1b87bf516265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:29.369371 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b899dd-2a13-4d76-ad1b-1b87bf516265 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:29.369684 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b899dd-2a13-4d76-ad1b-1b87bf516265 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:29.369976 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b899dd-2a13-4d76-ad1b-1b87bf516265 admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:29.370672 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b899dd-2a13-4d76-ad1b-1b87bf516265 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:29.372625 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b899dd-2a13-4d76-ad1b-1b87bf516265 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:29.404454 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1180/2361] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:25:29 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:25:29.408797 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d9ed1-a9da-4e14-8fa5-eab2f4d9051f None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:29.421173 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d9ed1-a9da-4e14-8fa5-eab2f4d9051f None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:29.452803 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:29.486914 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11b28a-1a05-4a2b-8a54-a7f0cef7be05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:29.487639 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11b28a-1a05-4a2b-8a54-a7f0cef7be05 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:29.487928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11b28a-1a05-4a2b-8a54-a7f0cef7be05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:29.488574 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11b28a-1a05-4a2b-8a54-a7f0cef7be05 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:29.488922 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11b28a-1a05-4a2b-8a54-a7f0cef7be05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:29.490964 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11b28a-1a05-4a2b-8a54-a7f0cef7be05 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:29.515613 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1182/2362] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:25:29 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:25:29.516851 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b899dd-2a13-4d76-ad1b-1b87bf516265 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:29.527963 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b899dd-2a13-4d76-ad1b-1b87bf516265 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:29.532791 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ef644-5a0d-42ee-a1ee-5909c825f353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:29.533668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ef644-5a0d-42ee-a1ee-5909c825f353 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:29.533825 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ef644-5a0d-42ee-a1ee-5909c825f353 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:29.534098 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ef644-5a0d-42ee-a1ee-5909c825f353 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:29.534966 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558ef644-5a0d-42ee-a1ee-5909c825f353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:29.536610 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558ef644-5a0d-42ee-a1ee-5909c825f353 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:29.548441 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1181/2363] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:25:29.552521 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11b28a-1a05-4a2b-8a54-a7f0cef7be05 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:29.562756 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11b28a-1a05-4a2b-8a54-a7f0cef7be05 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:29.567442 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b793407-0419-46ce-9ab0-10618a018038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:29.568116 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b793407-0419-46ce-9ab0-10618a018038 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:29.568404 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b793407-0419-46ce-9ab0-10618a018038 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:29.568690 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b793407-0419-46ce-9ab0-10618a018038 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:29.569431 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b793407-0419-46ce-9ab0-10618a018038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:29.571103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b793407-0419-46ce-9ab0-10618a018038 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:29.581053 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1183/2364] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:25:29.585415 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ef644-5a0d-42ee-a1ee-5909c825f353 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:29.600428 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ef644-5a0d-42ee-a1ee-5909c825f353 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:29.609368 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418a319-b56c-4f36-8607-f06a2cbb04b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:29.611040 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418a319-b56c-4f36-8607-f06a2cbb04b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:29.611451 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418a319-b56c-4f36-8607-f06a2cbb04b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:29.611905 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418a319-b56c-4f36-8607-f06a2cbb04b0 admin admin] PATH_INFO: `/v3/projects/cd0475087cae46d1bed735898579523f/users/eae3242e1f1440829e51bb94025b2ba2/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:29.628139 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418a319-b56c-4f36-8607-f06a2cbb04b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd0475087cae46d1bed735898579523f, user_id=eae3242e1f1440829e51bb94025b2ba2, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:29.630815 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418a319-b56c-4f36-8607-f06a2cbb04b0 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:29.645398 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1182/2365] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:25:29 2022] PUT /identity/v3/projects/cd0475087cae46d1bed735898579523f/users/eae3242e1f1440829e51bb94025b2ba2/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:25:29.656580 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf67c2-e780-4d62-ae44-07fdea2a73d9 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:29.656929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf67c2-e780-4d62-ae44-07fdea2a73d9 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:29.657284 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf67c2-e780-4d62-ae44-07fdea2a73d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:29.691217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-8ecf67c2-e780-4d62-ae44-07fdea2a73d9 None None] MFA Rules not processed for user `eae3242e1f1440829e51bb94025b2ba2`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:29.692372 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-8ecf67c2-e780-4d62-ae44-07fdea2a73d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:29.770501 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1184/2366] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:25:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:25:29.775274 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418a319-b56c-4f36-8607-f06a2cbb04b0 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:29.838065 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418a319-b56c-4f36-8607-f06a2cbb04b0 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:29.843344 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3275ed-3251-49c5-8565-a32356a9d3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:29.844014 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3275ed-3251-49c5-8565-a32356a9d3d6 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:29.844326 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3275ed-3251-49c5-8565-a32356a9d3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:29.844606 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3275ed-3251-49c5-8565-a32356a9d3d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:29.845302 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3275ed-3251-49c5-8565-a32356a9d3d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:29.847344 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3275ed-3251-49c5-8565-a32356a9d3d6 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:29.873462 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1183/2367] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:25:29 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:25:29.877764 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf67c2-e780-4d62-ae44-07fdea2a73d9 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:29.894950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf67c2-e780-4d62-ae44-07fdea2a73d9 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:29.957912 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94522c21-f7b8-4471-9f60-ed777d26412f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:29.958548 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94522c21-f7b8-4471-9f60-ed777d26412f admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:29.958837 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94522c21-f7b8-4471-9f60-ed777d26412f admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:29.959146 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94522c21-f7b8-4471-9f60-ed777d26412f admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:29.959846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94522c21-f7b8-4471-9f60-ed777d26412f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:29.961780 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94522c21-f7b8-4471-9f60-ed777d26412f admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:29.983548 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1185/2368] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:25:29 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:25:29.988607 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3275ed-3251-49c5-8565-a32356a9d3d6 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:30.004056 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3275ed-3251-49c5-8565-a32356a9d3d6 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:30.009677 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a5dd3-7bec-499a-b83f-398d43021b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:30.010344 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a5dd3-7bec-499a-b83f-398d43021b53 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:30.010625 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a5dd3-7bec-499a-b83f-398d43021b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:30.010898 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a5dd3-7bec-499a-b83f-398d43021b53 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:30.011669 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1a5dd3-7bec-499a-b83f-398d43021b53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:30.013313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1a5dd3-7bec-499a-b83f-398d43021b53 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:30.024381 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1184/2369] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:25:30.028679 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94522c21-f7b8-4471-9f60-ed777d26412f admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:30.039498 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94522c21-f7b8-4471-9f60-ed777d26412f admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:30.045287 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90ae7e-505b-4d2c-b14e-bdd8ea4ac11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:30.046182 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90ae7e-505b-4d2c-b14e-bdd8ea4ac11f admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:30.046467 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90ae7e-505b-4d2c-b14e-bdd8ea4ac11f admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:30.046771 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90ae7e-505b-4d2c-b14e-bdd8ea4ac11f admin admin] PATH_INFO: `/v3/projects/da3ac5edb21842f3ba7c19488f2c46c1/users/1ec8455bd62e4afcb2bce999e9ef7610/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:30.062271 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f90ae7e-505b-4d2c-b14e-bdd8ea4ac11f admin admin] RBAC: Authorizing `identity:create_grant(project_id=da3ac5edb21842f3ba7c19488f2c46c1, user_id=1ec8455bd62e4afcb2bce999e9ef7610, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:30.065013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f90ae7e-505b-4d2c-b14e-bdd8ea4ac11f admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:30.079641 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1186/2370] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:25:30 2022] PUT /identity/v3/projects/da3ac5edb21842f3ba7c19488f2c46c1/users/1ec8455bd62e4afcb2bce999e9ef7610/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:25:30.083771 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a5dd3-7bec-499a-b83f-398d43021b53 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:30.094048 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a5dd3-7bec-499a-b83f-398d43021b53 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:30.121767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:30.156430 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd735e-7d27-4d92-a4ac-9fc1eb31b437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:30.157082 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd735e-7d27-4d92-a4ac-9fc1eb31b437 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:30.157380 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd735e-7d27-4d92-a4ac-9fc1eb31b437 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:30.157661 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd735e-7d27-4d92-a4ac-9fc1eb31b437 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:30.158398 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd735e-7d27-4d92-a4ac-9fc1eb31b437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:30.160045 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd735e-7d27-4d92-a4ac-9fc1eb31b437 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:30.169903 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1185/2371] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:25:30.174394 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90ae7e-505b-4d2c-b14e-bdd8ea4ac11f admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:30.181364 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:30.186748 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90ae7e-505b-4d2c-b14e-bdd8ea4ac11f admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:30.197422 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea8d18-25e6-483d-a5e4-78cd2102d8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:30.198668 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea8d18-25e6-483d-a5e4-78cd2102d8ca admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:30.199015 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea8d18-25e6-483d-a5e4-78cd2102d8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:30.199411 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea8d18-25e6-483d-a5e4-78cd2102d8ca admin admin] PATH_INFO: `/v3/projects/da3ac5edb21842f3ba7c19488f2c46c1/users/1ec8455bd62e4afcb2bce999e9ef7610/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:30.202521 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea8d18-25e6-483d-a5e4-78cd2102d8ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=da3ac5edb21842f3ba7c19488f2c46c1, user_id=1ec8455bd62e4afcb2bce999e9ef7610, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:30.206449 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea8d18-25e6-483d-a5e4-78cd2102d8ca admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:30.225273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1187/2372] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:25:30 2022] PUT /identity/v3/projects/da3ac5edb21842f3ba7c19488f2c46c1/users/1ec8455bd62e4afcb2bce999e9ef7610/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:25:30.236627 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a5d9d-5bf7-44c9-8a54-506cbb8efa41 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:30.236943 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a5d9d-5bf7-44c9-8a54-506cbb8efa41 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:30.237221 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a5d9d-5bf7-44c9-8a54-506cbb8efa41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:30.279386 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-0e5a5d9d-5bf7-44c9-8a54-506cbb8efa41 None None] MFA Rules not processed for user `1ec8455bd62e4afcb2bce999e9ef7610`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:30.281277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-0e5a5d9d-5bf7-44c9-8a54-506cbb8efa41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:30.376387 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1186/2373] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:25:30 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:25:30.388178 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea8d18-25e6-483d-a5e4-78cd2102d8ca admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:30.389913 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-eaea8d18-25e6-483d-a5e4-78cd2102d8ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:30.449958 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea8d18-25e6-483d-a5e4-78cd2102d8ca admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:30.453835 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f4d60-5ee8-4411-b766-65e8913e3fa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:30.454488 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f4d60-5ee8-4411-b766-65e8913e3fa6 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:30.454774 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f4d60-5ee8-4411-b766-65e8913e3fa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:30.455066 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f4d60-5ee8-4411-b766-65e8913e3fa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:30.465003 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f4d60-5ee8-4411-b766-65e8913e3fa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:30.466444 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f4d60-5ee8-4411-b766-65e8913e3fa6 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:30.477170 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1188/2374] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:25:30 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:25:30.521496 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a5d9d-5bf7-44c9-8a54-506cbb8efa41 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:30.523021 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-0e5a5d9d-5bf7-44c9-8a54-506cbb8efa41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:30.535698 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a5d9d-5bf7-44c9-8a54-506cbb8efa41 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:30.540378 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4883fcbd-863f-4edc-ac99-9ef3f42e9c22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:30.541028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883fcbd-863f-4edc-ac99-9ef3f42e9c22 service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:30.541309 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883fcbd-863f-4edc-ac99-9ef3f42e9c22 service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:30.541582 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883fcbd-863f-4edc-ac99-9ef3f42e9c22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:30.605950 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4883fcbd-863f-4edc-ac99-9ef3f42e9c22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:30.608214 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4883fcbd-863f-4edc-ac99-9ef3f42e9c22 service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:30.623026 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1187/2375] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:25:30 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:25:37.776030 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f4d60-5ee8-4411-b766-65e8913e3fa6 service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:37.840621 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f4d60-5ee8-4411-b766-65e8913e3fa6 service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:37.845553 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ac813-6cb8-4e27-9028-44c2b93db09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:37.846645 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ac813-6cb8-4e27-9028-44c2b93db09a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:37.847246 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ac813-6cb8-4e27-9028-44c2b93db09a admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:37.847411 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ac813-6cb8-4e27-9028-44c2b93db09a admin admin] PATH_INFO: `/v3/users/ca06c3fab691475184f461c4954a7d7d` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:37.848504 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ac813-6cb8-4e27-9028-44c2b93db09a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca06c3fab691475184f461c4954a7d7d)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:37.850505 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ac813-6cb8-4e27-9028-44c2b93db09a admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:37.901345 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-d60ac813-6cb8-4e27-9028-44c2b93db09a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca06c3fab691475184f461c4954a7d7d'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:37.915116 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-d60ac813-6cb8-4e27-9028-44c2b93db09a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca06c3fab691475184f461c4954a7d7d'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:37.915847 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-d60ac813-6cb8-4e27-9028-44c2b93db09a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca06c3fab691475184f461c4954a7d7d'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:37.916460 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-d60ac813-6cb8-4e27-9028-44c2b93db09a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca06c3fab691475184f461c4954a7d7d'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:37.923951 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-d60ac813-6cb8-4e27-9028-44c2b93db09a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca06c3fab691475184f461c4954a7d7d'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:37.944464 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1189/2376] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:25:37 2022] DELETE /identity/v3/users/ca06c3fab691475184f461c4954a7d7d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:25:37.949145 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bdbc9a-27a5-40f9-88d8-bcb6b130e4e3 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:37.949677 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bdbc9a-27a5-40f9-88d8-bcb6b130e4e3 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:37.949815 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bdbc9a-27a5-40f9-88d8-bcb6b130e4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:37.967284 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-e5bdbc9a-27a5-40f9-88d8-bcb6b130e4e3 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:37.968511 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e5bdbc9a-27a5-40f9-88d8-bcb6b130e4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:38.013687 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:38.031151 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1188/2377] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:25:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:25:38.041583 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ac813-6cb8-4e27-9028-44c2b93db09a admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:38.043145 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-d60ac813-6cb8-4e27-9028-44c2b93db09a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:38.058493 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:38.106451 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ac813-6cb8-4e27-9028-44c2b93db09a admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:38.110884 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f12ab-c381-4c9f-ba6c-c5f35e1816d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:38.111575 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f12ab-c381-4c9f-ba6c-c5f35e1816d8 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:38.111861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f12ab-c381-4c9f-ba6c-c5f35e1816d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:38.112139 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f12ab-c381-4c9f-ba6c-c5f35e1816d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:38.121764 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f12ab-c381-4c9f-ba6c-c5f35e1816d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:38.123236 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f12ab-c381-4c9f-ba6c-c5f35e1816d8 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:38.133144 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1190/2378] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:25:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:25:38.409827 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bdbc9a-27a5-40f9-88d8-bcb6b130e4e3 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:38.411342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e5bdbc9a-27a5-40f9-88d8-bcb6b130e4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:38.426627 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bdbc9a-27a5-40f9-88d8-bcb6b130e4e3 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:38.432406 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51d811-7c77-48f5-9d5f-b7bdfb0a513f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:38.433298 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51d811-7c77-48f5-9d5f-b7bdfb0a513f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:38.433588 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51d811-7c77-48f5-9d5f-b7bdfb0a513f admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:38.433869 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51d811-7c77-48f5-9d5f-b7bdfb0a513f admin admin] PATH_INFO: `/v3/projects/eb8017f7f7ff42079a53548e60ed2d9d` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:38.435294 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e51d811-7c77-48f5-9d5f-b7bdfb0a513f admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb8017f7f7ff42079a53548e60ed2d9d)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:38.437262 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e51d811-7c77-48f5-9d5f-b7bdfb0a513f admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:38.488971 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-1e51d811-7c77-48f5-9d5f-b7bdfb0a513f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb8017f7f7ff42079a53548e60ed2d9d', 'request_id': 'req-1e51d811-7c77-48f5-9d5f-b7bdfb0a513f'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:38.494975 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-1e51d811-7c77-48f5-9d5f-b7bdfb0a513f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb8017f7f7ff42079a53548e60ed2d9d', 'request_id': 'req-1e51d811-7c77-48f5-9d5f-b7bdfb0a513f'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:38.501219 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-1e51d811-7c77-48f5-9d5f-b7bdfb0a513f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb8017f7f7ff42079a53548e60ed2d9d', 'request_id': 'req-1e51d811-7c77-48f5-9d5f-b7bdfb0a513f'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:38.515540 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-1e51d811-7c77-48f5-9d5f-b7bdfb0a513f admin admin] The token cache is being invalidate because project eb8017f7f7ff42079a53548e60ed2d9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:25:38.515857 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-1e51d811-7c77-48f5-9d5f-b7bdfb0a513f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:38.516573 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-1e51d811-7c77-48f5-9d5f-b7bdfb0a513f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:38.518221 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1189/2379] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:25:38 2022] DELETE /identity/v3/projects/eb8017f7f7ff42079a53548e60ed2d9d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:25:38.522396 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f12ab-c381-4c9f-ba6c-c5f35e1816d8 service neutron] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:38.532412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f12ab-c381-4c9f-ba6c-c5f35e1816d8 service neutron] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:38.592677 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b70e1-8a5b-404e-95db-33e5fa00f5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:38.593554 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b70e1-8a5b-404e-95db-33e5fa00f5d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:38.593838 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b70e1-8a5b-404e-95db-33e5fa00f5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:38.594116 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b70e1-8a5b-404e-95db-33e5fa00f5d0 admin admin] PATH_INFO: `/v3/users/eae3242e1f1440829e51bb94025b2ba2` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:38.595415 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925b70e1-8a5b-404e-95db-33e5fa00f5d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eae3242e1f1440829e51bb94025b2ba2)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:38.597355 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925b70e1-8a5b-404e-95db-33e5fa00f5d0 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:38.649069 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-925b70e1-8a5b-404e-95db-33e5fa00f5d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eae3242e1f1440829e51bb94025b2ba2'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:38.660528 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-925b70e1-8a5b-404e-95db-33e5fa00f5d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eae3242e1f1440829e51bb94025b2ba2'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:38.661180 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-925b70e1-8a5b-404e-95db-33e5fa00f5d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eae3242e1f1440829e51bb94025b2ba2'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:38.661781 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-925b70e1-8a5b-404e-95db-33e5fa00f5d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eae3242e1f1440829e51bb94025b2ba2'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:38.668419 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-925b70e1-8a5b-404e-95db-33e5fa00f5d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eae3242e1f1440829e51bb94025b2ba2'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:38.687076 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1191/2380] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:25:38 2022] DELETE /identity/v3/users/eae3242e1f1440829e51bb94025b2ba2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:25:38.982951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51d811-7c77-48f5-9d5f-b7bdfb0a513f admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:38.984460 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-1e51d811-7c77-48f5-9d5f-b7bdfb0a513f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:39.000906 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51d811-7c77-48f5-9d5f-b7bdfb0a513f admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:39.062804 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b21f36-12d4-4434-a92d-9a84b43f2147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:39.063698 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b21f36-12d4-4434-a92d-9a84b43f2147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:39.063985 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b21f36-12d4-4434-a92d-9a84b43f2147 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:39.064263 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b21f36-12d4-4434-a92d-9a84b43f2147 admin admin] PATH_INFO: `/v3/projects/cd0475087cae46d1bed735898579523f` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:39.065578 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b21f36-12d4-4434-a92d-9a84b43f2147 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd0475087cae46d1bed735898579523f)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:39.067545 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b21f36-12d4-4434-a92d-9a84b43f2147 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:39.116086 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-40b21f36-12d4-4434-a92d-9a84b43f2147 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd0475087cae46d1bed735898579523f', 'request_id': 'req-40b21f36-12d4-4434-a92d-9a84b43f2147'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:39.121722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-40b21f36-12d4-4434-a92d-9a84b43f2147 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd0475087cae46d1bed735898579523f', 'request_id': 'req-40b21f36-12d4-4434-a92d-9a84b43f2147'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:39.128311 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-40b21f36-12d4-4434-a92d-9a84b43f2147 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd0475087cae46d1bed735898579523f', 'request_id': 'req-40b21f36-12d4-4434-a92d-9a84b43f2147'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:39.141506 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-40b21f36-12d4-4434-a92d-9a84b43f2147 admin admin] The token cache is being invalidate because project cd0475087cae46d1bed735898579523f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:25:39.141818 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-40b21f36-12d4-4434-a92d-9a84b43f2147 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:39.142492 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-40b21f36-12d4-4434-a92d-9a84b43f2147 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:39.144242 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1190/2381] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:25:38 2022] DELETE /identity/v3/projects/cd0475087cae46d1bed735898579523f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:25:39.148448 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b70e1-8a5b-404e-95db-33e5fa00f5d0 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:39.149968 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-925b70e1-8a5b-404e-95db-33e5fa00f5d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:39.162936 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b70e1-8a5b-404e-95db-33e5fa00f5d0 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:39.195942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:39.221653 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91eca5a-e0af-4627-b613-78412a378cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:39.222556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91eca5a-e0af-4627-b613-78412a378cd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:39.222865 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91eca5a-e0af-4627-b613-78412a378cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:39.223178 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91eca5a-e0af-4627-b613-78412a378cd4 admin admin] PATH_INFO: `/v3/users/1ec8455bd62e4afcb2bce999e9ef7610` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:39.224410 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91eca5a-e0af-4627-b613-78412a378cd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ec8455bd62e4afcb2bce999e9ef7610)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:39.226323 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91eca5a-e0af-4627-b613-78412a378cd4 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:39.273610 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-e91eca5a-e0af-4627-b613-78412a378cd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ec8455bd62e4afcb2bce999e9ef7610'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:39.285917 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-e91eca5a-e0af-4627-b613-78412a378cd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ec8455bd62e4afcb2bce999e9ef7610'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:39.286669 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-e91eca5a-e0af-4627-b613-78412a378cd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ec8455bd62e4afcb2bce999e9ef7610'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:39.287341 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-e91eca5a-e0af-4627-b613-78412a378cd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ec8455bd62e4afcb2bce999e9ef7610'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:39.294778 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-e91eca5a-e0af-4627-b613-78412a378cd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ec8455bd62e4afcb2bce999e9ef7610'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:39.309216 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1192/2382] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:25:39 2022] DELETE /identity/v3/users/1ec8455bd62e4afcb2bce999e9ef7610 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:25:39.593277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b21f36-12d4-4434-a92d-9a84b43f2147 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:39.594775 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-40b21f36-12d4-4434-a92d-9a84b43f2147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:39.602811 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:39.609605 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b21f36-12d4-4434-a92d-9a84b43f2147 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:39.670716 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b19749-cb89-4d97-a472-dc6e89e87a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:39.671630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b19749-cb89-4d97-a472-dc6e89e87a42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:39.671914 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b19749-cb89-4d97-a472-dc6e89e87a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:39.672191 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b19749-cb89-4d97-a472-dc6e89e87a42 admin admin] PATH_INFO: `/v3/projects/da3ac5edb21842f3ba7c19488f2c46c1` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:39.673394 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b19749-cb89-4d97-a472-dc6e89e87a42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da3ac5edb21842f3ba7c19488f2c46c1)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:39.675383 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b19749-cb89-4d97-a472-dc6e89e87a42 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:39.723463 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-87b19749-cb89-4d97-a472-dc6e89e87a42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da3ac5edb21842f3ba7c19488f2c46c1', 'request_id': 'req-87b19749-cb89-4d97-a472-dc6e89e87a42'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:39.730930 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-87b19749-cb89-4d97-a472-dc6e89e87a42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da3ac5edb21842f3ba7c19488f2c46c1', 'request_id': 'req-87b19749-cb89-4d97-a472-dc6e89e87a42'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:39.740240 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-87b19749-cb89-4d97-a472-dc6e89e87a42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da3ac5edb21842f3ba7c19488f2c46c1', 'request_id': 'req-87b19749-cb89-4d97-a472-dc6e89e87a42'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:39.755682 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-87b19749-cb89-4d97-a472-dc6e89e87a42 admin admin] The token cache is being invalidate because project da3ac5edb21842f3ba7c19488f2c46c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:25:39.755978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-87b19749-cb89-4d97-a472-dc6e89e87a42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:39.756619 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-87b19749-cb89-4d97-a472-dc6e89e87a42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:39.758193 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1191/2383] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:25:39 2022] DELETE /identity/v3/projects/da3ac5edb21842f3ba7c19488f2c46c1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:25:39.764356 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593e9be-3bcc-40b6-8863-68937a310231 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:39.764649 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593e9be-3bcc-40b6-8863-68937a310231 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:39.764931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593e9be-3bcc-40b6-8863-68937a310231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:39.783416 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-0593e9be-3bcc-40b6-8863-68937a310231 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:39.784635 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-0593e9be-3bcc-40b6-8863-68937a310231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:39.855882 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1193/2384] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:25:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:25:39.866512 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7457e10-fc77-4c75-8a9c-5997806f574b None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:39.866804 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7457e10-fc77-4c75-8a9c-5997806f574b None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:39.867109 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7457e10-fc77-4c75-8a9c-5997806f574b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:39.888120 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-b7457e10-fc77-4c75-8a9c-5997806f574b None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:39.889331 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-b7457e10-fc77-4c75-8a9c-5997806f574b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:39.898040 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1192/2385] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:25:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:25:39.902613 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593e9be-3bcc-40b6-8863-68937a310231 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:39.913783 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593e9be-3bcc-40b6-8863-68937a310231 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:39.921370 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ad503-1088-4176-94f8-3e5d273a6c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:39.922239 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ad503-1088-4176-94f8-3e5d273a6c1f admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:39.922664 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ad503-1088-4176-94f8-3e5d273a6c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:39.923027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ad503-1088-4176-94f8-3e5d273a6c1f admin admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:39.924232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ad503-1088-4176-94f8-3e5d273a6c1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:25:39.924548 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ad503-1088-4176-94f8-3e5d273a6c1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:39.926165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ad503-1088-4176-94f8-3e5d273a6c1f admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:39.938765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1194/2386] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 15:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:25:39.943484 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7457e10-fc77-4c75-8a9c-5997806f574b None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:39.953122 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7457e10-fc77-4c75-8a9c-5997806f574b None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:39.958986 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20398adb-39ba-420b-95cc-2e06dcf5d262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:39.959685 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20398adb-39ba-420b-95cc-2e06dcf5d262 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:39.959974 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20398adb-39ba-420b-95cc-2e06dcf5d262 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:39.960256 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20398adb-39ba-420b-95cc-2e06dcf5d262 admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:39.960955 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20398adb-39ba-420b-95cc-2e06dcf5d262 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:39.962931 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20398adb-39ba-420b-95cc-2e06dcf5d262 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:39.994363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1193/2387] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:25:39 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:25:39.996445 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ad503-1088-4176-94f8-3e5d273a6c1f admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:40.009207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ad503-1088-4176-94f8-3e5d273a6c1f admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:40.050749 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:40.076832 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962d8a3-d597-4ffc-b92a-05ff32b25fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:40.077811 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962d8a3-d597-4ffc-b92a-05ff32b25fce admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:40.078186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962d8a3-d597-4ffc-b92a-05ff32b25fce admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:40.078570 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962d8a3-d597-4ffc-b92a-05ff32b25fce admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:40.079631 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f962d8a3-d597-4ffc-b92a-05ff32b25fce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:40.081850 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f962d8a3-d597-4ffc-b92a-05ff32b25fce admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:40.105770 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1195/2388] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:25:39 2022] POST /identity/v3/users => generated 294 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:25:40.110255 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20398adb-39ba-420b-95cc-2e06dcf5d262 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:40.121028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20398adb-39ba-420b-95cc-2e06dcf5d262 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:40.126244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36c8c4-10e0-4dee-b80a-bc503bf5692a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:40.127316 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36c8c4-10e0-4dee-b80a-bc503bf5692a admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:40.127748 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36c8c4-10e0-4dee-b80a-bc503bf5692a admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:40.128117 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36c8c4-10e0-4dee-b80a-bc503bf5692a admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:40.128866 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36c8c4-10e0-4dee-b80a-bc503bf5692a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:40.131494 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36c8c4-10e0-4dee-b80a-bc503bf5692a admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:40.143173 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1194/2389] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:25:40.147369 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962d8a3-d597-4ffc-b92a-05ff32b25fce admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:40.157638 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962d8a3-d597-4ffc-b92a-05ff32b25fce admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:40.162845 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c718edef-2e5f-4896-a0a4-4eea2bd27994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:40.163555 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718edef-2e5f-4896-a0a4-4eea2bd27994 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:40.163841 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718edef-2e5f-4896-a0a4-4eea2bd27994 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:40.164121 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718edef-2e5f-4896-a0a4-4eea2bd27994 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:40.164863 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c718edef-2e5f-4896-a0a4-4eea2bd27994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:40.166467 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c718edef-2e5f-4896-a0a4-4eea2bd27994 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:40.176962 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1196/2390] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:25:40.181027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36c8c4-10e0-4dee-b80a-bc503bf5692a admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:40.191229 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36c8c4-10e0-4dee-b80a-bc503bf5692a admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:40.196014 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1472f05-34ff-4b90-9a89-1a7d9972f0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:40.196911 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1472f05-34ff-4b90-9a89-1a7d9972f0ac admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:40.197197 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1472f05-34ff-4b90-9a89-1a7d9972f0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:40.197480 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1472f05-34ff-4b90-9a89-1a7d9972f0ac admin admin] PATH_INFO: `/v3/projects/a2199a8584a846069d284d07e016bd01/users/8cdcee4639a547bab7221f5425109750/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:40.212926 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1472f05-34ff-4b90-9a89-1a7d9972f0ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2199a8584a846069d284d07e016bd01, user_id=8cdcee4639a547bab7221f5425109750, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:40.215648 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1472f05-34ff-4b90-9a89-1a7d9972f0ac admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:40.230833 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1195/2391] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:25:40 2022] PUT /identity/v3/projects/a2199a8584a846069d284d07e016bd01/users/8cdcee4639a547bab7221f5425109750/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:25:40.243337 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca80050-1fd2-4811-bbb8-2b371af11d3a None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:40.243652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca80050-1fd2-4811-bbb8-2b371af11d3a None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:40.243989 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca80050-1fd2-4811-bbb8-2b371af11d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:40.276003 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-5ca80050-1fd2-4811-bbb8-2b371af11d3a None None] MFA Rules not processed for user `8cdcee4639a547bab7221f5425109750`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:40.277234 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-5ca80050-1fd2-4811-bbb8-2b371af11d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:40.344328 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1197/2392] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:25:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:25:40.349198 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1472f05-34ff-4b90-9a89-1a7d9972f0ac admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:40.381641 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:40.414660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1472f05-34ff-4b90-9a89-1a7d9972f0ac admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:40.419873 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc9272-c547-4a2e-a88e-8888c64e28b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:40.420535 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc9272-c547-4a2e-a88e-8888c64e28b8 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:40.420820 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc9272-c547-4a2e-a88e-8888c64e28b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:40.421101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc9272-c547-4a2e-a88e-8888c64e28b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:40.421794 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc9272-c547-4a2e-a88e-8888c64e28b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:40.423769 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc9272-c547-4a2e-a88e-8888c64e28b8 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:40.454034 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1196/2393] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:25:40 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:25:40.460243 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca80050-1fd2-4811-bbb8-2b371af11d3a None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:40.471505 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca80050-1fd2-4811-bbb8-2b371af11d3a None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:40.538746 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1760751d-7b79-4ec3-85b3-ec372de66980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:40.539512 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760751d-7b79-4ec3-85b3-ec372de66980 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:40.539840 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760751d-7b79-4ec3-85b3-ec372de66980 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:40.540155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760751d-7b79-4ec3-85b3-ec372de66980 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:40.540949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1760751d-7b79-4ec3-85b3-ec372de66980 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:40.542867 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1760751d-7b79-4ec3-85b3-ec372de66980 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:40.565134 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1198/2394] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:25:40 2022] POST /identity/v3/users => generated 299 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:25:40.569380 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc9272-c547-4a2e-a88e-8888c64e28b8 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:40.580378 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc9272-c547-4a2e-a88e-8888c64e28b8 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:40.585349 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddca06a-b845-4a4f-885a-5912a85dc139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:40.586018 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddca06a-b845-4a4f-885a-5912a85dc139 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:40.586306 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddca06a-b845-4a4f-885a-5912a85dc139 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:40.586589 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddca06a-b845-4a4f-885a-5912a85dc139 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:40.587377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddca06a-b845-4a4f-885a-5912a85dc139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:40.589021 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddca06a-b845-4a4f-885a-5912a85dc139 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:40.599734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1197/2395] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:25:40.603974 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1760751d-7b79-4ec3-85b3-ec372de66980 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:40.614838 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1760751d-7b79-4ec3-85b3-ec372de66980 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:40.620746 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2fb26-ad55-45a4-b48b-357f3a9ee7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:40.621637 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2fb26-ad55-45a4-b48b-357f3a9ee7f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:40.621922 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2fb26-ad55-45a4-b48b-357f3a9ee7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:40.622197 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2fb26-ad55-45a4-b48b-357f3a9ee7f6 admin admin] PATH_INFO: `/v3/projects/580d45256d094dfb90f2ec56a70e4065/users/bcb7c9a8c1b0436da21e7990d3231d0d/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:40.638624 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f2fb26-ad55-45a4-b48b-357f3a9ee7f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=580d45256d094dfb90f2ec56a70e4065, user_id=bcb7c9a8c1b0436da21e7990d3231d0d, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:40.641417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f2fb26-ad55-45a4-b48b-357f3a9ee7f6 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:40.657001 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1199/2396] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:25:40 2022] PUT /identity/v3/projects/580d45256d094dfb90f2ec56a70e4065/users/bcb7c9a8c1b0436da21e7990d3231d0d/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:25:40.661546 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddca06a-b845-4a4f-885a-5912a85dc139 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:40.676595 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddca06a-b845-4a4f-885a-5912a85dc139 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:40.738306 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3176c8-3c62-44f9-a1fd-0d72b7eed96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:40.738897 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3176c8-3c62-44f9-a1fd-0d72b7eed96f admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:40.739267 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3176c8-3c62-44f9-a1fd-0d72b7eed96f admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:40.739549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3176c8-3c62-44f9-a1fd-0d72b7eed96f admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:40.740434 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3176c8-3c62-44f9-a1fd-0d72b7eed96f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:40.742057 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3176c8-3c62-44f9-a1fd-0d72b7eed96f admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:40.752658 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1198/2397] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:25:40.757213 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2fb26-ad55-45a4-b48b-357f3a9ee7f6 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:40.769984 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2fb26-ad55-45a4-b48b-357f3a9ee7f6 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:40.778359 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d75e69-6489-4751-b21a-938c748b9aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:40.779290 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d75e69-6489-4751-b21a-938c748b9aaa admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:40.779576 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d75e69-6489-4751-b21a-938c748b9aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:40.779852 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d75e69-6489-4751-b21a-938c748b9aaa admin admin] PATH_INFO: `/v3/projects/580d45256d094dfb90f2ec56a70e4065/users/bcb7c9a8c1b0436da21e7990d3231d0d/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:40.781834 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d75e69-6489-4751-b21a-938c748b9aaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=580d45256d094dfb90f2ec56a70e4065, user_id=bcb7c9a8c1b0436da21e7990d3231d0d, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:40.784588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d75e69-6489-4751-b21a-938c748b9aaa admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:40.796005 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1200/2398] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:25:40 2022] PUT /identity/v3/projects/580d45256d094dfb90f2ec56a70e4065/users/bcb7c9a8c1b0436da21e7990d3231d0d/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:25:40.806406 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f5c93-e442-4732-b7d5-9be9c314884d None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:40.806690 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f5c93-e442-4732-b7d5-9be9c314884d None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:40.806965 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f5c93-e442-4732-b7d5-9be9c314884d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:40.834407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-052f5c93-e442-4732-b7d5-9be9c314884d None None] MFA Rules not processed for user `bcb7c9a8c1b0436da21e7990d3231d0d`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:40.835642 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-052f5c93-e442-4732-b7d5-9be9c314884d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:40.905319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1199/2399] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:25:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:25:40.916678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d75e69-6489-4751-b21a-938c748b9aaa admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:40.918812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-39d75e69-6489-4751-b21a-938c748b9aaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:40.925031 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:40.981955 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d75e69-6489-4751-b21a-938c748b9aaa admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:40.986223 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b5c42-ba07-401c-b78c-a4bde3e0d371 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:40.986874 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b5c42-ba07-401c-b78c-a4bde3e0d371 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:40.987195 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b5c42-ba07-401c-b78c-a4bde3e0d371 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:40.987478 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b5c42-ba07-401c-b78c-a4bde3e0d371 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:40.997370 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b5c42-ba07-401c-b78c-a4bde3e0d371 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:40.998736 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b5c42-ba07-401c-b78c-a4bde3e0d371 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:41.008765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1201/2400] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:25:40 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:25:41.766373 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052f5c93-e442-4732-b7d5-9be9c314884d None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:41.799770 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:41.836317 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052f5c93-e442-4732-b7d5-9be9c314884d None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:41.841606 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a80ca-379c-4af7-9225-6c239bbacd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:41.842488 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a80ca-379c-4af7-9225-6c239bbacd04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:41.842775 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a80ca-379c-4af7-9225-6c239bbacd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:41.843069 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a80ca-379c-4af7-9225-6c239bbacd04 admin admin] PATH_INFO: `/v3/users/8cdcee4639a547bab7221f5425109750` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:41.844299 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77a80ca-379c-4af7-9225-6c239bbacd04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cdcee4639a547bab7221f5425109750)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:41.846325 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77a80ca-379c-4af7-9225-6c239bbacd04 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:41.895637 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-c77a80ca-379c-4af7-9225-6c239bbacd04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cdcee4639a547bab7221f5425109750'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:41.909288 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-c77a80ca-379c-4af7-9225-6c239bbacd04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cdcee4639a547bab7221f5425109750'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:41.909966 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-c77a80ca-379c-4af7-9225-6c239bbacd04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cdcee4639a547bab7221f5425109750'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:41.910590 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-c77a80ca-379c-4af7-9225-6c239bbacd04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cdcee4639a547bab7221f5425109750'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:41.917710 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-c77a80ca-379c-4af7-9225-6c239bbacd04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cdcee4639a547bab7221f5425109750'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:41.932529 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1200/2401] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:25:41 2022] DELETE /identity/v3/users/8cdcee4639a547bab7221f5425109750 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:25:41.938133 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59e655-dd82-4c96-917d-18bbaf44da4a None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:41.938974 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59e655-dd82-4c96-917d-18bbaf44da4a None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:41.939344 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59e655-dd82-4c96-917d-18bbaf44da4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:41.957367 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-2e59e655-dd82-4c96-917d-18bbaf44da4a None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:41.958572 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-2e59e655-dd82-4c96-917d-18bbaf44da4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:42.036475 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1202/2402] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:25:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:25:42.046595 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a80ca-379c-4af7-9225-6c239bbacd04 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:42.048279 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-c77a80ca-379c-4af7-9225-6c239bbacd04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:42.116207 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a80ca-379c-4af7-9225-6c239bbacd04 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:42.120834 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc0012-fcef-4ca9-b8ba-01192bafe9dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:42.121747 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc0012-fcef-4ca9-b8ba-01192bafe9dd service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:42.122079 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc0012-fcef-4ca9-b8ba-01192bafe9dd service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:42.122363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc0012-fcef-4ca9-b8ba-01192bafe9dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:42.133220 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc0012-fcef-4ca9-b8ba-01192bafe9dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:42.134666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc0012-fcef-4ca9-b8ba-01192bafe9dd service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:42.145114 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1201/2403] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:25:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:25:42.429785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59e655-dd82-4c96-917d-18bbaf44da4a None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:42.431383 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-2e59e655-dd82-4c96-917d-18bbaf44da4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:42.443570 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59e655-dd82-4c96-917d-18bbaf44da4a None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:42.449038 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a789814-63de-414b-903d-bceca33a42c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:42.449592 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a789814-63de-414b-903d-bceca33a42c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:42.449894 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a789814-63de-414b-903d-bceca33a42c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:42.450131 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a789814-63de-414b-903d-bceca33a42c2 admin admin] PATH_INFO: `/v3/projects/a2199a8584a846069d284d07e016bd01` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:42.451668 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a789814-63de-414b-903d-bceca33a42c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2199a8584a846069d284d07e016bd01)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:42.453590 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a789814-63de-414b-903d-bceca33a42c2 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:42.504983 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1a789814-63de-414b-903d-bceca33a42c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2199a8584a846069d284d07e016bd01', 'request_id': 'req-1a789814-63de-414b-903d-bceca33a42c2'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:42.510708 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1a789814-63de-414b-903d-bceca33a42c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2199a8584a846069d284d07e016bd01', 'request_id': 'req-1a789814-63de-414b-903d-bceca33a42c2'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:42.519786 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1a789814-63de-414b-903d-bceca33a42c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2199a8584a846069d284d07e016bd01', 'request_id': 'req-1a789814-63de-414b-903d-bceca33a42c2'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:42.532826 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1a789814-63de-414b-903d-bceca33a42c2 admin admin] The token cache is being invalidate because project a2199a8584a846069d284d07e016bd01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:25:42.533110 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1a789814-63de-414b-903d-bceca33a42c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:42.533912 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1a789814-63de-414b-903d-bceca33a42c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:42.535638 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1203/2404] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:25:42 2022] DELETE /identity/v3/projects/a2199a8584a846069d284d07e016bd01 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:25:42.539948 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc0012-fcef-4ca9-b8ba-01192bafe9dd service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:42.550723 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc0012-fcef-4ca9-b8ba-01192bafe9dd service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:42.594950 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:42.609842 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caff491-e919-4224-91d6-4776e819dd08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:42.611041 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caff491-e919-4224-91d6-4776e819dd08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:42.611376 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caff491-e919-4224-91d6-4776e819dd08 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:42.611647 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caff491-e919-4224-91d6-4776e819dd08 admin admin] PATH_INFO: `/v3/users/bcb7c9a8c1b0436da21e7990d3231d0d` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:42.612848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caff491-e919-4224-91d6-4776e819dd08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcb7c9a8c1b0436da21e7990d3231d0d)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:42.614777 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caff491-e919-4224-91d6-4776e819dd08 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:42.661915 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-2caff491-e919-4224-91d6-4776e819dd08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcb7c9a8c1b0436da21e7990d3231d0d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:42.673705 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-2caff491-e919-4224-91d6-4776e819dd08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcb7c9a8c1b0436da21e7990d3231d0d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:42.674395 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-2caff491-e919-4224-91d6-4776e819dd08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcb7c9a8c1b0436da21e7990d3231d0d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:42.675023 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-2caff491-e919-4224-91d6-4776e819dd08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcb7c9a8c1b0436da21e7990d3231d0d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:42.681920 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-2caff491-e919-4224-91d6-4776e819dd08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcb7c9a8c1b0436da21e7990d3231d0d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:42.696426 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1202/2405] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:25:42 2022] DELETE /identity/v3/users/bcb7c9a8c1b0436da21e7990d3231d0d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:25:42.976552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a789814-63de-414b-903d-bceca33a42c2 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:42.978156 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-1a789814-63de-414b-903d-bceca33a42c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:42.992703 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a789814-63de-414b-903d-bceca33a42c2 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:43.020496 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:43.051603 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb974932-f0d8-47a3-bae7-a2231694ad9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:43.052477 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb974932-f0d8-47a3-bae7-a2231694ad9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:43.052756 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb974932-f0d8-47a3-bae7-a2231694ad9a admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:43.053026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb974932-f0d8-47a3-bae7-a2231694ad9a admin admin] PATH_INFO: `/v3/projects/580d45256d094dfb90f2ec56a70e4065` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:43.054202 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb974932-f0d8-47a3-bae7-a2231694ad9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=580d45256d094dfb90f2ec56a70e4065)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:43.056173 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb974932-f0d8-47a3-bae7-a2231694ad9a admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:43.104331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fb974932-f0d8-47a3-bae7-a2231694ad9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '580d45256d094dfb90f2ec56a70e4065', 'request_id': 'req-fb974932-f0d8-47a3-bae7-a2231694ad9a'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:43.111080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fb974932-f0d8-47a3-bae7-a2231694ad9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '580d45256d094dfb90f2ec56a70e4065', 'request_id': 'req-fb974932-f0d8-47a3-bae7-a2231694ad9a'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:43.121428 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fb974932-f0d8-47a3-bae7-a2231694ad9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '580d45256d094dfb90f2ec56a70e4065', 'request_id': 'req-fb974932-f0d8-47a3-bae7-a2231694ad9a'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:43.133181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fb974932-f0d8-47a3-bae7-a2231694ad9a admin admin] The token cache is being invalidate because project 580d45256d094dfb90f2ec56a70e4065 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:25:43.133495 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fb974932-f0d8-47a3-bae7-a2231694ad9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:43.134244 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fb974932-f0d8-47a3-bae7-a2231694ad9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:25:43.136056 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1204/2406] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:25:42 2022] DELETE /identity/v3/projects/580d45256d094dfb90f2ec56a70e4065 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:25:43.141181 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eae1a1-84e5-4976-9b5a-eb99d555a176 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:43.141688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eae1a1-84e5-4976-9b5a-eb99d555a176 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:43.141814 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eae1a1-84e5-4976-9b5a-eb99d555a176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:43.160488 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-55eae1a1-84e5-4976-9b5a-eb99d555a176 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:43.161690 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-55eae1a1-84e5-4976-9b5a-eb99d555a176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:43.235279 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1203/2407] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:25:43.243618 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde1a5a-e53c-429c-a84c-2c3001719faf None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:43.243911 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde1a5a-e53c-429c-a84c-2c3001719faf None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:43.244186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde1a5a-e53c-429c-a84c-2c3001719faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:43.263576 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-bcde1a5a-e53c-429c-a84c-2c3001719faf None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:43.264784 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-bcde1a5a-e53c-429c-a84c-2c3001719faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:43.273740 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1205/2408] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:25:43.277935 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eae1a1-84e5-4976-9b5a-eb99d555a176 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:43.288618 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eae1a1-84e5-4976-9b5a-eb99d555a176 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:43.294887 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df56e1-9690-4478-98e3-c4209ac7ed28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:43.295592 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df56e1-9690-4478-98e3-c4209ac7ed28 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:43.295875 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df56e1-9690-4478-98e3-c4209ac7ed28 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:43.296152 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df56e1-9690-4478-98e3-c4209ac7ed28 admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:43.296917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df56e1-9690-4478-98e3-c4209ac7ed28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:25:43.297236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df56e1-9690-4478-98e3-c4209ac7ed28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:43.298882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df56e1-9690-4478-98e3-c4209ac7ed28 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:43.311504 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1204/2409] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 15:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:25:43.315907 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde1a5a-e53c-429c-a84c-2c3001719faf None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:43.325627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde1a5a-e53c-429c-a84c-2c3001719faf None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:43.331334 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b672e4c-48d2-49fc-b7fd-358f5d27caaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:43.331996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b672e4c-48d2-49fc-b7fd-358f5d27caaa admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:43.332280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b672e4c-48d2-49fc-b7fd-358f5d27caaa admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:43.332557 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b672e4c-48d2-49fc-b7fd-358f5d27caaa admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:43.333253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b672e4c-48d2-49fc-b7fd-358f5d27caaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:43.335196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b672e4c-48d2-49fc-b7fd-358f5d27caaa admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:43.367805 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1206/2410] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:25:43 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:25:43.372457 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df56e1-9690-4478-98e3-c4209ac7ed28 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:43.384775 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df56e1-9690-4478-98e3-c4209ac7ed28 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:43.435365 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:43.462774 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e090ffba-0307-4d56-b1c3-993072415763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:43.463520 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090ffba-0307-4d56-b1c3-993072415763 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:43.463807 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090ffba-0307-4d56-b1c3-993072415763 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:43.464083 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090ffba-0307-4d56-b1c3-993072415763 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:43.469092 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e090ffba-0307-4d56-b1c3-993072415763 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:43.469092 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e090ffba-0307-4d56-b1c3-993072415763 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:43.498497 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1205/2411] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:25:43 2022] POST /identity/v3/users => generated 287 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:25:43.503218 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b672e4c-48d2-49fc-b7fd-358f5d27caaa admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:43.514058 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b672e4c-48d2-49fc-b7fd-358f5d27caaa admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:43.519090 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466bc15-392f-46fd-8e79-690aa3e4a017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:43.519946 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466bc15-392f-46fd-8e79-690aa3e4a017 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:43.520232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466bc15-392f-46fd-8e79-690aa3e4a017 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:43.520512 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466bc15-392f-46fd-8e79-690aa3e4a017 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:43.521239 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0466bc15-392f-46fd-8e79-690aa3e4a017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:43.522841 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0466bc15-392f-46fd-8e79-690aa3e4a017 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:43.533545 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1207/2412] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:25:43.538095 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e090ffba-0307-4d56-b1c3-993072415763 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:43.555838 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e090ffba-0307-4d56-b1c3-993072415763 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:43.564880 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3442b77c-f86f-496e-b75f-b88139632aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:43.565563 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442b77c-f86f-496e-b75f-b88139632aa9 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:43.565852 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442b77c-f86f-496e-b75f-b88139632aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:43.566124 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442b77c-f86f-496e-b75f-b88139632aa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:43.566860 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3442b77c-f86f-496e-b75f-b88139632aa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:43.568671 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3442b77c-f86f-496e-b75f-b88139632aa9 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:43.580631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1206/2413] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:25:43.585048 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466bc15-392f-46fd-8e79-690aa3e4a017 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:43.595919 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466bc15-392f-46fd-8e79-690aa3e4a017 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:43.601861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85a48e-7340-42f5-9c8d-3ad13290163a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:43.602754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85a48e-7340-42f5-9c8d-3ad13290163a admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:43.603036 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85a48e-7340-42f5-9c8d-3ad13290163a admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:43.603386 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85a48e-7340-42f5-9c8d-3ad13290163a admin admin] PATH_INFO: `/v3/projects/a1a059f868fe495fb26aedbd8a817631/users/61d061999f294bbe9a399f5157a47bab/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:43.619090 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85a48e-7340-42f5-9c8d-3ad13290163a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1a059f868fe495fb26aedbd8a817631, user_id=61d061999f294bbe9a399f5157a47bab, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:43.621754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85a48e-7340-42f5-9c8d-3ad13290163a admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:43.637136 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1208/2414] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:25:43 2022] PUT /identity/v3/projects/a1a059f868fe495fb26aedbd8a817631/users/61d061999f294bbe9a399f5157a47bab/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:25:43.642284 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbf0b4-16ed-495c-8164-97b2d74e8ecb None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:43.642674 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbf0b4-16ed-495c-8164-97b2d74e8ecb None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:43.642965 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbf0b4-16ed-495c-8164-97b2d74e8ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:43.660469 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-b5dbf0b4-16ed-495c-8164-97b2d74e8ecb None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:43.661708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-b5dbf0b4-16ed-495c-8164-97b2d74e8ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:43.724602 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1207/2415] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:25:43.734606 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85a48e-7340-42f5-9c8d-3ad13290163a admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:43.736933 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-ee85a48e-7340-42f5-9c8d-3ad13290163a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:43.827905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85a48e-7340-42f5-9c8d-3ad13290163a admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:43.831930 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4089e9-70e0-47d1-9714-4235bbf92725 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:43.832577 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4089e9-70e0-47d1-9714-4235bbf92725 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:43.832858 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4089e9-70e0-47d1-9714-4235bbf92725 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:43.833131 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4089e9-70e0-47d1-9714-4235bbf92725 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:43.842237 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4089e9-70e0-47d1-9714-4235bbf92725 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:43.843714 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4089e9-70e0-47d1-9714-4235bbf92725 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:43.847571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:43.854962 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1209/2416] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:25:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:25:44.524356 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6a648-24d4-4444-aa4b-673cf3490594 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:44.524874 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6a648-24d4-4444-aa4b-673cf3490594 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:44.524988 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6a648-24d4-4444-aa4b-673cf3490594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:44.554163 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-2eb6a648-24d4-4444-aa4b-673cf3490594 None None] MFA Rules not processed for user `61d061999f294bbe9a399f5157a47bab`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:44.555564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-2eb6a648-24d4-4444-aa4b-673cf3490594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:44.623784 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1208/2417] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:25:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:25:44.628564 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4089e9-70e0-47d1-9714-4235bbf92725 service neutron] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:44.638804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4089e9-70e0-47d1-9714-4235bbf92725 service neutron] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:44.643855 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b613e-3b88-425f-96a1-cdc294965ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:44.644579 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b613e-3b88-425f-96a1-cdc294965ff9 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:44.644812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b613e-3b88-425f-96a1-cdc294965ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:44.645088 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b613e-3b88-425f-96a1-cdc294965ff9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:44.645824 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b613e-3b88-425f-96a1-cdc294965ff9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:44.647785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b613e-3b88-425f-96a1-cdc294965ff9 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:44.679088 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1210/2418] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:25:44 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:25:44.682203 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb6a648-24d4-4444-aa4b-673cf3490594 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:44.693295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb6a648-24d4-4444-aa4b-673cf3490594 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:44.751492 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0007757-0256-4d72-bbd8-d8d31d13e509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:44.752289 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0007757-0256-4d72-bbd8-d8d31d13e509 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:44.752576 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0007757-0256-4d72-bbd8-d8d31d13e509 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:44.752859 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0007757-0256-4d72-bbd8-d8d31d13e509 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:44.753577 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0007757-0256-4d72-bbd8-d8d31d13e509 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:44.755588 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0007757-0256-4d72-bbd8-d8d31d13e509 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:44.763160 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:44.775784 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1209/2419] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:25:44 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:25:44.779929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b613e-3b88-425f-96a1-cdc294965ff9 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:44.790724 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b613e-3b88-425f-96a1-cdc294965ff9 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:44.795496 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443c908-4a2f-4a8c-914b-eb5e8bbaac0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:44.796154 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443c908-4a2f-4a8c-914b-eb5e8bbaac0b admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:44.796435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443c908-4a2f-4a8c-914b-eb5e8bbaac0b admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:44.796712 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443c908-4a2f-4a8c-914b-eb5e8bbaac0b admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:44.797437 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1443c908-4a2f-4a8c-914b-eb5e8bbaac0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:44.799119 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1443c908-4a2f-4a8c-914b-eb5e8bbaac0b admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:44.809946 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1211/2420] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:25:44.814254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0007757-0256-4d72-bbd8-d8d31d13e509 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:44.824953 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0007757-0256-4d72-bbd8-d8d31d13e509 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:44.830120 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d80968b-b964-45c4-8656-a552c99b3aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:44.831009 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80968b-b964-45c4-8656-a552c99b3aa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:44.831325 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80968b-b964-45c4-8656-a552c99b3aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:44.831606 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80968b-b964-45c4-8656-a552c99b3aa1 admin admin] PATH_INFO: `/v3/projects/88f75a8f64e34e6a9e619d4a93a4892e/users/1959c067b89b44a8b273b72fb75bb64e/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:44.846579 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d80968b-b964-45c4-8656-a552c99b3aa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88f75a8f64e34e6a9e619d4a93a4892e, user_id=1959c067b89b44a8b273b72fb75bb64e, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:44.849296 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d80968b-b964-45c4-8656-a552c99b3aa1 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:44.864723 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1210/2421] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:25:44 2022] PUT /identity/v3/projects/88f75a8f64e34e6a9e619d4a93a4892e/users/1959c067b89b44a8b273b72fb75bb64e/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:25:44.868805 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443c908-4a2f-4a8c-914b-eb5e8bbaac0b admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:44.879520 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443c908-4a2f-4a8c-914b-eb5e8bbaac0b admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:44.936655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f0bb5-17bc-4a03-9fb2-f0a60c069fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:44.937311 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f0bb5-17bc-4a03-9fb2-f0a60c069fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:44.937602 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f0bb5-17bc-4a03-9fb2-f0a60c069fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:44.937885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f0bb5-17bc-4a03-9fb2-f0a60c069fc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:44.938621 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f0bb5-17bc-4a03-9fb2-f0a60c069fc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:44.940343 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f0bb5-17bc-4a03-9fb2-f0a60c069fc4 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:44.950737 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1212/2422] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:25:44.955212 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d80968b-b964-45c4-8656-a552c99b3aa1 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:44.966316 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d80968b-b964-45c4-8656-a552c99b3aa1 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:44.971494 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9ecba-fbfc-4a84-9ed9-3a790a48541f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:44.972397 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9ecba-fbfc-4a84-9ed9-3a790a48541f admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:44.972719 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9ecba-fbfc-4a84-9ed9-3a790a48541f admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:44.973003 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9ecba-fbfc-4a84-9ed9-3a790a48541f admin admin] PATH_INFO: `/v3/projects/88f75a8f64e34e6a9e619d4a93a4892e/users/1959c067b89b44a8b273b72fb75bb64e/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:44.975020 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a9ecba-fbfc-4a84-9ed9-3a790a48541f admin admin] RBAC: Authorizing `identity:create_grant(project_id=88f75a8f64e34e6a9e619d4a93a4892e, user_id=1959c067b89b44a8b273b72fb75bb64e, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:44.977711 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a9ecba-fbfc-4a84-9ed9-3a790a48541f admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:44.988466 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1211/2423] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:25:44 2022] PUT /identity/v3/projects/88f75a8f64e34e6a9e619d4a93a4892e/users/1959c067b89b44a8b273b72fb75bb64e/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:25:45.665042 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99d4f1-4c2a-43ba-9d5c-2f8c3cecb94b None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:45.665644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99d4f1-4c2a-43ba-9d5c-2f8c3cecb94b None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:45.665644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99d4f1-4c2a-43ba-9d5c-2f8c3cecb94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:45.692763 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-5e99d4f1-4c2a-43ba-9d5c-2f8c3cecb94b None None] MFA Rules not processed for user `1959c067b89b44a8b273b72fb75bb64e`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:45.694801 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-5e99d4f1-4c2a-43ba-9d5c-2f8c3cecb94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:45.765694 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1213/2424] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:25:45 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:25:45.777271 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9ecba-fbfc-4a84-9ed9-3a790a48541f admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:45.779291 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-78a9ecba-fbfc-4a84-9ed9-3a790a48541f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:45.838133 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9ecba-fbfc-4a84-9ed9-3a790a48541f admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:45.843840 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55aebc-a5e5-4560-906a-b1e577ff18b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:45.844738 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55aebc-a5e5-4560-906a-b1e577ff18b2 service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:45.845188 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55aebc-a5e5-4560-906a-b1e577ff18b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:45.845629 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55aebc-a5e5-4560-906a-b1e577ff18b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:45.915666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec55aebc-a5e5-4560-906a-b1e577ff18b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:45.917126 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec55aebc-a5e5-4560-906a-b1e577ff18b2 service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:45.926893 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1212/2425] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:25:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:25:47.291343 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99d4f1-4c2a-43ba-9d5c-2f8c3cecb94b None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:47.301748 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99d4f1-4c2a-43ba-9d5c-2f8c3cecb94b None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:47.305796 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651423e-fcb3-4b8c-84b6-c9b6759efae2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:47.306450 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651423e-fcb3-4b8c-84b6-c9b6759efae2 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:47.306731 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651423e-fcb3-4b8c-84b6-c9b6759efae2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:47.307002 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651423e-fcb3-4b8c-84b6-c9b6759efae2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:47.317222 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e651423e-fcb3-4b8c-84b6-c9b6759efae2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:47.318666 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e651423e-fcb3-4b8c-84b6-c9b6759efae2 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:47.328644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1214/2426] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:25:47 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:25:55.922438 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:INSERT=4,keystone:SELECT=65 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:57.324049 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=84,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:06.288148 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55aebc-a5e5-4560-906a-b1e577ff18b2 service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:06.289987 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-ec55aebc-a5e5-4560-906a-b1e577ff18b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:06.370425 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55aebc-a5e5-4560-906a-b1e577ff18b2 service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:06.378004 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4179cdb6-00c1-4af7-b5d1-45126aca40cb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb857f32bbc04d82bfa681bd5659765f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:06.378842 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179cdb6-00c1-4af7-b5d1-45126aca40cb service placement] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:06.379178 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179cdb6-00c1-4af7-b5d1-45126aca40cb service placement] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:06.379464 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179cdb6-00c1-4af7-b5d1-45126aca40cb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:06.381632 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-4179cdb6-00c1-4af7-b5d1-45126aca40cb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:06.393995 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4179cdb6-00c1-4af7-b5d1-45126aca40cb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:06.395659 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4179cdb6-00c1-4af7-b5d1-45126aca40cb service placement] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:06.406704 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1213/2427] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:26:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:26:06.873505 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651423e-fcb3-4b8c-84b6-c9b6759efae2 service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:06.947407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651423e-fcb3-4b8c-84b6-c9b6759efae2 service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:06.954513 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6e3cc-65af-47fe-bdf8-e3332fff28a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:06.955462 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6e3cc-65af-47fe-bdf8-e3332fff28a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:06.955750 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6e3cc-65af-47fe-bdf8-e3332fff28a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:06.956029 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6e3cc-65af-47fe-bdf8-e3332fff28a3 admin admin] PATH_INFO: `/v3/users/61d061999f294bbe9a399f5157a47bab` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:06.957377 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6e3cc-65af-47fe-bdf8-e3332fff28a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61d061999f294bbe9a399f5157a47bab)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:06.959412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6e3cc-65af-47fe-bdf8-e3332fff28a3 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:07.009409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-2bc6e3cc-65af-47fe-bdf8-e3332fff28a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61d061999f294bbe9a399f5157a47bab'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:07.021897 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-2bc6e3cc-65af-47fe-bdf8-e3332fff28a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61d061999f294bbe9a399f5157a47bab'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:07.022606 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-2bc6e3cc-65af-47fe-bdf8-e3332fff28a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61d061999f294bbe9a399f5157a47bab'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:07.023292 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-2bc6e3cc-65af-47fe-bdf8-e3332fff28a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61d061999f294bbe9a399f5157a47bab'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:07.030677 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-2bc6e3cc-65af-47fe-bdf8-e3332fff28a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61d061999f294bbe9a399f5157a47bab'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:07.045403 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1215/2428] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:26:06 2022] DELETE /identity/v3/users/61d061999f294bbe9a399f5157a47bab => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:26:07.244444 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4179cdb6-00c1-4af7-b5d1-45126aca40cb service placement] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:07.256984 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4179cdb6-00c1-4af7-b5d1-45126aca40cb service placement] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:07.316508 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cff84c-4af6-4384-b084-5c0a7222721a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:07.317319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cff84c-4af6-4384-b084-5c0a7222721a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:07.317603 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cff84c-4af6-4384-b084-5c0a7222721a admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:07.317881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cff84c-4af6-4384-b084-5c0a7222721a admin admin] PATH_INFO: `/v3/projects/a1a059f868fe495fb26aedbd8a817631` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:07.319106 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cff84c-4af6-4384-b084-5c0a7222721a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1a059f868fe495fb26aedbd8a817631)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:07.321036 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cff84c-4af6-4384-b084-5c0a7222721a admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:07.375829 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-b7cff84c-4af6-4384-b084-5c0a7222721a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1a059f868fe495fb26aedbd8a817631', 'request_id': 'req-b7cff84c-4af6-4384-b084-5c0a7222721a'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:07.382188 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-b7cff84c-4af6-4384-b084-5c0a7222721a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1a059f868fe495fb26aedbd8a817631', 'request_id': 'req-b7cff84c-4af6-4384-b084-5c0a7222721a'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:07.388998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-b7cff84c-4af6-4384-b084-5c0a7222721a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1a059f868fe495fb26aedbd8a817631', 'request_id': 'req-b7cff84c-4af6-4384-b084-5c0a7222721a'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:07.404208 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-b7cff84c-4af6-4384-b084-5c0a7222721a admin admin] The token cache is being invalidate because project a1a059f868fe495fb26aedbd8a817631 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:26:07.404526 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-b7cff84c-4af6-4384-b084-5c0a7222721a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:07.405304 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-b7cff84c-4af6-4384-b084-5c0a7222721a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:07.406953 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1214/2429] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:26:07 2022] DELETE /identity/v3/projects/a1a059f868fe495fb26aedbd8a817631 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:26:07.411337 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6e3cc-65af-47fe-bdf8-e3332fff28a3 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:07.412937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-2bc6e3cc-65af-47fe-bdf8-e3332fff28a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:07.426944 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6e3cc-65af-47fe-bdf8-e3332fff28a3 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:07.498686 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e644bf2-e4e5-4f9e-af19-1f817a979a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:07.499625 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e644bf2-e4e5-4f9e-af19-1f817a979a32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:07.499913 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e644bf2-e4e5-4f9e-af19-1f817a979a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:07.500193 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e644bf2-e4e5-4f9e-af19-1f817a979a32 admin admin] PATH_INFO: `/v3/users/1959c067b89b44a8b273b72fb75bb64e` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:07.501753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e644bf2-e4e5-4f9e-af19-1f817a979a32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1959c067b89b44a8b273b72fb75bb64e)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:07.503748 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e644bf2-e4e5-4f9e-af19-1f817a979a32 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:07.526369 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:07.553857 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-4e644bf2-e4e5-4f9e-af19-1f817a979a32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1959c067b89b44a8b273b72fb75bb64e'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:07.565879 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-4e644bf2-e4e5-4f9e-af19-1f817a979a32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1959c067b89b44a8b273b72fb75bb64e'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:07.566625 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-4e644bf2-e4e5-4f9e-af19-1f817a979a32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1959c067b89b44a8b273b72fb75bb64e'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:07.567305 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-4e644bf2-e4e5-4f9e-af19-1f817a979a32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1959c067b89b44a8b273b72fb75bb64e'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:07.574406 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-4e644bf2-e4e5-4f9e-af19-1f817a979a32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1959c067b89b44a8b273b72fb75bb64e'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:07.591802 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1216/2430] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:26:07 2022] DELETE /identity/v3/users/1959c067b89b44a8b273b72fb75bb64e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:26:07.784839 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cff84c-4af6-4384-b084-5c0a7222721a admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:07.786313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-b7cff84c-4af6-4384-b084-5c0a7222721a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:07.799663 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cff84c-4af6-4384-b084-5c0a7222721a admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:07.820147 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:07.863909 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d509d-1ed1-42e1-b052-ea3b54fd0c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:07.864796 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d509d-1ed1-42e1-b052-ea3b54fd0c66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:07.865092 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d509d-1ed1-42e1-b052-ea3b54fd0c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:07.865369 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d509d-1ed1-42e1-b052-ea3b54fd0c66 admin admin] PATH_INFO: `/v3/projects/88f75a8f64e34e6a9e619d4a93a4892e` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:07.866587 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635d509d-1ed1-42e1-b052-ea3b54fd0c66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88f75a8f64e34e6a9e619d4a93a4892e)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:07.868563 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635d509d-1ed1-42e1-b052-ea3b54fd0c66 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:07.915957 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-635d509d-1ed1-42e1-b052-ea3b54fd0c66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88f75a8f64e34e6a9e619d4a93a4892e', 'request_id': 'req-635d509d-1ed1-42e1-b052-ea3b54fd0c66'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:07.921647 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-635d509d-1ed1-42e1-b052-ea3b54fd0c66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88f75a8f64e34e6a9e619d4a93a4892e', 'request_id': 'req-635d509d-1ed1-42e1-b052-ea3b54fd0c66'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:07.927974 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-635d509d-1ed1-42e1-b052-ea3b54fd0c66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88f75a8f64e34e6a9e619d4a93a4892e', 'request_id': 'req-635d509d-1ed1-42e1-b052-ea3b54fd0c66'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:07.942064 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-635d509d-1ed1-42e1-b052-ea3b54fd0c66 admin admin] The token cache is being invalidate because project 88f75a8f64e34e6a9e619d4a93a4892e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:26:07.942470 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-635d509d-1ed1-42e1-b052-ea3b54fd0c66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:07.943142 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-635d509d-1ed1-42e1-b052-ea3b54fd0c66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:07.944679 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1215/2431] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:26:07 2022] DELETE /identity/v3/projects/88f75a8f64e34e6a9e619d4a93a4892e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:26:07.950735 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079ffd5-af24-44ed-8e89-352d28b74d05 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:07.951271 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079ffd5-af24-44ed-8e89-352d28b74d05 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:07.951395 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079ffd5-af24-44ed-8e89-352d28b74d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:07.970546 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-6079ffd5-af24-44ed-8e89-352d28b74d05 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:26:07.971982 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-6079ffd5-af24-44ed-8e89-352d28b74d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:08.032522 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1217/2432] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:26:08.040704 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9de9d3-9b64-4175-9003-fa58e2a0f443 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:08.040997 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9de9d3-9b64-4175-9003-fa58e2a0f443 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:08.041277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9de9d3-9b64-4175-9003-fa58e2a0f443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:08.062011 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-aa9de9d3-9b64-4175-9003-fa58e2a0f443 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:26:08.063269 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-aa9de9d3-9b64-4175-9003-fa58e2a0f443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:08.075111 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1216/2433] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:26:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:26:08.079769 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6079ffd5-af24-44ed-8e89-352d28b74d05 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:08.094780 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6079ffd5-af24-44ed-8e89-352d28b74d05 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:08.100868 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79866a-e5dd-4457-808d-c15a098fc74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:08.101521 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79866a-e5dd-4457-808d-c15a098fc74a admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:08.101801 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79866a-e5dd-4457-808d-c15a098fc74a admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:08.102073 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79866a-e5dd-4457-808d-c15a098fc74a admin admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:08.102821 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79866a-e5dd-4457-808d-c15a098fc74a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:08.103150 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79866a-e5dd-4457-808d-c15a098fc74a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:08.104751 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79866a-e5dd-4457-808d-c15a098fc74a admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:08.117399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1218/2434] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 15:26:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:26:08.121830 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9de9d3-9b64-4175-9003-fa58e2a0f443 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:08.131796 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9de9d3-9b64-4175-9003-fa58e2a0f443 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:08.137911 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27499faf-4d17-408c-a214-e85e73aa7eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:08.138610 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27499faf-4d17-408c-a214-e85e73aa7eb8 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:08.138893 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27499faf-4d17-408c-a214-e85e73aa7eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:08.139204 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27499faf-4d17-408c-a214-e85e73aa7eb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:08.139902 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27499faf-4d17-408c-a214-e85e73aa7eb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:08.141814 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27499faf-4d17-408c-a214-e85e73aa7eb8 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:08.171347 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1217/2435] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:26:08 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:26:08.177605 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79866a-e5dd-4457-808d-c15a098fc74a admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:08.192131 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79866a-e5dd-4457-808d-c15a098fc74a admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:08.252254 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9652b-c821-4772-8e97-cc5593fd32cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:08.252945 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9652b-c821-4772-8e97-cc5593fd32cb admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:08.253240 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9652b-c821-4772-8e97-cc5593fd32cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:08.253522 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9652b-c821-4772-8e97-cc5593fd32cb admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:08.254215 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a9652b-c821-4772-8e97-cc5593fd32cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:08.256186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a9652b-c821-4772-8e97-cc5593fd32cb admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:08.279198 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1219/2436] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:26:08 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:26:08.283770 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27499faf-4d17-408c-a214-e85e73aa7eb8 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:08.294360 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27499faf-4d17-408c-a214-e85e73aa7eb8 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:08.299396 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68fd38b-7678-49f4-bb22-556aa65f9483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:08.300066 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fd38b-7678-49f4-bb22-556aa65f9483 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:08.300353 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fd38b-7678-49f4-bb22-556aa65f9483 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:08.300666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fd38b-7678-49f4-bb22-556aa65f9483 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:08.301432 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68fd38b-7678-49f4-bb22-556aa65f9483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:08.303073 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68fd38b-7678-49f4-bb22-556aa65f9483 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:08.313990 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1218/2437] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:26:08.318645 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9652b-c821-4772-8e97-cc5593fd32cb admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:08.332529 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9652b-c821-4772-8e97-cc5593fd32cb admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:08.337636 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef3ad19-a983-4f97-94df-99f0a203469d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:08.338291 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3ad19-a983-4f97-94df-99f0a203469d admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:08.338575 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3ad19-a983-4f97-94df-99f0a203469d admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:08.338849 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3ad19-a983-4f97-94df-99f0a203469d admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:08.339615 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef3ad19-a983-4f97-94df-99f0a203469d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:08.341214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef3ad19-a983-4f97-94df-99f0a203469d admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:08.351873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1220/2438] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:26:08.356164 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68fd38b-7678-49f4-bb22-556aa65f9483 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:08.368522 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68fd38b-7678-49f4-bb22-556aa65f9483 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:08.373853 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be89eff-9dd2-4464-abfc-06d432a40267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:08.374744 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be89eff-9dd2-4464-abfc-06d432a40267 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:08.375067 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be89eff-9dd2-4464-abfc-06d432a40267 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:08.375366 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be89eff-9dd2-4464-abfc-06d432a40267 admin admin] PATH_INFO: `/v3/projects/7b99ef7e6b8d43e2a06661021656d400/users/6d72866ac24e40a38b58902f9a0184a2/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:08.390034 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be89eff-9dd2-4464-abfc-06d432a40267 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b99ef7e6b8d43e2a06661021656d400, user_id=6d72866ac24e40a38b58902f9a0184a2, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:08.392989 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be89eff-9dd2-4464-abfc-06d432a40267 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:08.414070 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1219/2439] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:26:08 2022] PUT /identity/v3/projects/7b99ef7e6b8d43e2a06661021656d400/users/6d72866ac24e40a38b58902f9a0184a2/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:26:08.427943 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ee6e5-a608-4f3a-af22-2c2bdcc0dc3b None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:08.428235 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ee6e5-a608-4f3a-af22-2c2bdcc0dc3b None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:08.428518 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ee6e5-a608-4f3a-af22-2c2bdcc0dc3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:08.435959 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:08.465706 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-078ee6e5-a608-4f3a-af22-2c2bdcc0dc3b None None] MFA Rules not processed for user `6d72866ac24e40a38b58902f9a0184a2`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:26:08.466951 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-078ee6e5-a608-4f3a-af22-2c2bdcc0dc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:08.534911 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1221/2440] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:26:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:26:08.539562 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be89eff-9dd2-4464-abfc-06d432a40267 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:08.590388 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:08.606704 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be89eff-9dd2-4464-abfc-06d432a40267 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:08.612764 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dffe36e-9fdb-450a-9cfe-42afa0911abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:08.613438 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffe36e-9fdb-450a-9cfe-42afa0911abd admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:08.613717 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffe36e-9fdb-450a-9cfe-42afa0911abd admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:08.614018 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffe36e-9fdb-450a-9cfe-42afa0911abd admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:08.614734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dffe36e-9fdb-450a-9cfe-42afa0911abd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:08.616736 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dffe36e-9fdb-450a-9cfe-42afa0911abd admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:08.643321 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1220/2441] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:26:08 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:26:08.649173 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ee6e5-a608-4f3a-af22-2c2bdcc0dc3b None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:08.660697 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ee6e5-a608-4f3a-af22-2c2bdcc0dc3b None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:08.721468 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d22b57-f82a-4e60-b560-236eebd86669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:08.722161 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d22b57-f82a-4e60-b560-236eebd86669 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:08.722443 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d22b57-f82a-4e60-b560-236eebd86669 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:08.722723 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d22b57-f82a-4e60-b560-236eebd86669 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:08.723474 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d22b57-f82a-4e60-b560-236eebd86669 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:08.725424 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d22b57-f82a-4e60-b560-236eebd86669 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:08.747365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1222/2442] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:26:08 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:26:08.751531 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dffe36e-9fdb-450a-9cfe-42afa0911abd admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:08.762612 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dffe36e-9fdb-450a-9cfe-42afa0911abd admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:08.768915 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aecb6b2-b984-42d8-86fb-c9f255f0eb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:08.769593 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aecb6b2-b984-42d8-86fb-c9f255f0eb73 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:08.769927 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aecb6b2-b984-42d8-86fb-c9f255f0eb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:08.770202 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aecb6b2-b984-42d8-86fb-c9f255f0eb73 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:08.770973 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aecb6b2-b984-42d8-86fb-c9f255f0eb73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:08.772628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aecb6b2-b984-42d8-86fb-c9f255f0eb73 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:08.784590 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1221/2443] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:26:08.788839 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d22b57-f82a-4e60-b560-236eebd86669 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:08.799849 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d22b57-f82a-4e60-b560-236eebd86669 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:08.810938 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd56b6-912b-4e9c-8443-0f322e164c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:08.811915 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd56b6-912b-4e9c-8443-0f322e164c93 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:08.812376 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd56b6-912b-4e9c-8443-0f322e164c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:08.812756 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd56b6-912b-4e9c-8443-0f322e164c93 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:08.813960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd56b6-912b-4e9c-8443-0f322e164c93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:08.815748 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd56b6-912b-4e9c-8443-0f322e164c93 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:08.826866 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1223/2444] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:26:08.831283 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aecb6b2-b984-42d8-86fb-c9f255f0eb73 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:08.841878 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aecb6b2-b984-42d8-86fb-c9f255f0eb73 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:08.847419 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e6c39-7684-4517-b4d1-7a7293750bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:08.848315 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e6c39-7684-4517-b4d1-7a7293750bbd admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:08.848599 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e6c39-7684-4517-b4d1-7a7293750bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:08.848880 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e6c39-7684-4517-b4d1-7a7293750bbd admin admin] PATH_INFO: `/v3/projects/20e845c95ac5437aaa11182d4ed6163e/users/955b16ce8b3247f1b530456013fbded1/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:08.865812 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e6c39-7684-4517-b4d1-7a7293750bbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=20e845c95ac5437aaa11182d4ed6163e, user_id=955b16ce8b3247f1b530456013fbded1, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:08.868626 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e6c39-7684-4517-b4d1-7a7293750bbd admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:08.882625 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1222/2445] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:26:08 2022] PUT /identity/v3/projects/20e845c95ac5437aaa11182d4ed6163e/users/955b16ce8b3247f1b530456013fbded1/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:26:08.899642 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbab0c7-2d6a-402a-89c0-cc3131159099 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:08.899935 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbab0c7-2d6a-402a-89c0-cc3131159099 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:08.900215 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbab0c7-2d6a-402a-89c0-cc3131159099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:08.928215 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-2bbab0c7-2d6a-402a-89c0-cc3131159099 None None] MFA Rules not processed for user `955b16ce8b3247f1b530456013fbded1`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:26:08.929459 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-2bbab0c7-2d6a-402a-89c0-cc3131159099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:09.012662 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1224/2446] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:26:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:26:09.018544 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e6c39-7684-4517-b4d1-7a7293750bbd admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:09.081496 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e6c39-7684-4517-b4d1-7a7293750bbd admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:09.086772 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4882da-9cdb-4047-abd9-5f400fa07b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:09.087497 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4882da-9cdb-4047-abd9-5f400fa07b7a admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:09.087784 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4882da-9cdb-4047-abd9-5f400fa07b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:09.088091 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4882da-9cdb-4047-abd9-5f400fa07b7a admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:09.088807 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4882da-9cdb-4047-abd9-5f400fa07b7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:09.090758 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4882da-9cdb-4047-abd9-5f400fa07b7a admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:09.119331 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1223/2447] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:26:09 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:26:09.123482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbab0c7-2d6a-402a-89c0-cc3131159099 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:09.129119 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:09.136355 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbab0c7-2d6a-402a-89c0-cc3131159099 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:09.196174 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4a14e-3859-4511-9cb6-ac97b109feb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:09.196881 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4a14e-3859-4511-9cb6-ac97b109feb6 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:09.197166 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4a14e-3859-4511-9cb6-ac97b109feb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:09.197444 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4a14e-3859-4511-9cb6-ac97b109feb6 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:09.198175 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c4a14e-3859-4511-9cb6-ac97b109feb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:09.200162 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c4a14e-3859-4511-9cb6-ac97b109feb6 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:09.220553 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1225/2448] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:26:09 2022] POST /identity/v3/users => generated 295 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:26:09.224633 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4882da-9cdb-4047-abd9-5f400fa07b7a admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:09.235944 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4882da-9cdb-4047-abd9-5f400fa07b7a admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:09.241855 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e3eed9-7aea-4829-940b-6d4f416817fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:09.242565 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3eed9-7aea-4829-940b-6d4f416817fd admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:09.242840 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3eed9-7aea-4829-940b-6d4f416817fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:09.243182 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3eed9-7aea-4829-940b-6d4f416817fd admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:09.243919 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e3eed9-7aea-4829-940b-6d4f416817fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:09.245537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e3eed9-7aea-4829-940b-6d4f416817fd admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:09.256676 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1224/2449] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:26:09.260790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4a14e-3859-4511-9cb6-ac97b109feb6 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:09.271440 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4a14e-3859-4511-9cb6-ac97b109feb6 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:09.276141 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589aba43-fcf4-4998-afa8-2650dede3bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:09.277035 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589aba43-fcf4-4998-afa8-2650dede3bad admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:09.277314 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589aba43-fcf4-4998-afa8-2650dede3bad admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:09.277591 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589aba43-fcf4-4998-afa8-2650dede3bad admin admin] PATH_INFO: `/v3/projects/fe2ad1bc4cd44516a59bfbebee205840/users/b77e4e4bb2a1419ba6e274187b64636e/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:09.292872 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589aba43-fcf4-4998-afa8-2650dede3bad admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe2ad1bc4cd44516a59bfbebee205840, user_id=b77e4e4bb2a1419ba6e274187b64636e, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:09.295586 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589aba43-fcf4-4998-afa8-2650dede3bad admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:09.310332 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1226/2450] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:26:09 2022] PUT /identity/v3/projects/fe2ad1bc4cd44516a59bfbebee205840/users/b77e4e4bb2a1419ba6e274187b64636e/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:26:09.314301 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e3eed9-7aea-4829-940b-6d4f416817fd admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:09.324865 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e3eed9-7aea-4829-940b-6d4f416817fd admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:09.381269 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93b952-6066-41c3-9889-5997bff70a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:09.381941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93b952-6066-41c3-9889-5997bff70a68 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:09.382114 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93b952-6066-41c3-9889-5997bff70a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:09.382420 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93b952-6066-41c3-9889-5997bff70a68 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:09.383175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93b952-6066-41c3-9889-5997bff70a68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:09.384794 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93b952-6066-41c3-9889-5997bff70a68 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:09.394560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1225/2451] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:26:09.399542 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589aba43-fcf4-4998-afa8-2650dede3bad admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:09.416305 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589aba43-fcf4-4998-afa8-2650dede3bad admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:09.424848 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ac0eb-6c68-4d00-8768-6b85645f39e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:09.425789 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ac0eb-6c68-4d00-8768-6b85645f39e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:09.426187 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ac0eb-6c68-4d00-8768-6b85645f39e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:09.426463 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ac0eb-6c68-4d00-8768-6b85645f39e4 admin admin] PATH_INFO: `/v3/projects/fe2ad1bc4cd44516a59bfbebee205840/users/b77e4e4bb2a1419ba6e274187b64636e/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:09.428897 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143ac0eb-6c68-4d00-8768-6b85645f39e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe2ad1bc4cd44516a59bfbebee205840, user_id=b77e4e4bb2a1419ba6e274187b64636e, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:09.431778 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143ac0eb-6c68-4d00-8768-6b85645f39e4 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:09.443297 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1227/2452] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:26:09 2022] PUT /identity/v3/projects/fe2ad1bc4cd44516a59bfbebee205840/users/b77e4e4bb2a1419ba6e274187b64636e/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:26:09.453944 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5bd764-a3b6-4a9a-accf-d2616f0b5431 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:09.454271 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5bd764-a3b6-4a9a-accf-d2616f0b5431 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:09.454558 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5bd764-a3b6-4a9a-accf-d2616f0b5431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:09.467822 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:09.486944 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-6a5bd764-a3b6-4a9a-accf-d2616f0b5431 None None] MFA Rules not processed for user `b77e4e4bb2a1419ba6e274187b64636e`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:26:09.488220 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-6a5bd764-a3b6-4a9a-accf-d2616f0b5431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:09.559416 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1226/2453] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:26:09 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:26:09.571267 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ac0eb-6c68-4d00-8768-6b85645f39e4 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:09.573136 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-143ac0eb-6c68-4d00-8768-6b85645f39e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:09.631190 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ac0eb-6c68-4d00-8768-6b85645f39e4 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:09.635376 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18107ec-3515-422a-8ac4-792954537e58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:09.635962 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18107ec-3515-422a-8ac4-792954537e58 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:09.636239 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18107ec-3515-422a-8ac4-792954537e58 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:09.636506 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18107ec-3515-422a-8ac4-792954537e58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:09.697257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18107ec-3515-422a-8ac4-792954537e58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:09.698691 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18107ec-3515-422a-8ac4-792954537e58 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:09.708656 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1228/2454] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:26:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:26:12.267080 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5bd764-a3b6-4a9a-accf-d2616f0b5431 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:12.277146 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5bd764-a3b6-4a9a-accf-d2616f0b5431 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:12.280856 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f98e24-2196-4048-be04-5d8aaf606ba9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:12.281532 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f98e24-2196-4048-be04-5d8aaf606ba9 service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:12.281820 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f98e24-2196-4048-be04-5d8aaf606ba9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:12.282099 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f98e24-2196-4048-be04-5d8aaf606ba9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:12.337660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f98e24-2196-4048-be04-5d8aaf606ba9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:12.339018 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f98e24-2196-4048-be04-5d8aaf606ba9 service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:12.348245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1227/2455] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:26:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:26:12.944140 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18107ec-3515-422a-8ac4-792954537e58 service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:12.953605 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18107ec-3515-422a-8ac4-792954537e58 service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:12.958117 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ab3ea-bd38-4000-ac89-45478f43dd27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:12.959091 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ab3ea-bd38-4000-ac89-45478f43dd27 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:12.959453 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ab3ea-bd38-4000-ac89-45478f43dd27 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:12.959799 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ab3ea-bd38-4000-ac89-45478f43dd27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:12.970055 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:12.972967 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953ab3ea-bd38-4000-ac89-45478f43dd27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:12.976465 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953ab3ea-bd38-4000-ac89-45478f43dd27 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:12.991277 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1229/2456] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:26:12 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:26:16.896905 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f98e24-2196-4048-be04-5d8aaf606ba9 service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:16.962367 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f98e24-2196-4048-be04-5d8aaf606ba9 service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:16.967672 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769836dc-d127-4c31-bd32-db169502a174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:16.968543 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769836dc-d127-4c31-bd32-db169502a174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:16.968832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769836dc-d127-4c31-bd32-db169502a174 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:16.969109 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769836dc-d127-4c31-bd32-db169502a174 admin admin] PATH_INFO: `/v3/users/6d72866ac24e40a38b58902f9a0184a2` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:16.970420 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769836dc-d127-4c31-bd32-db169502a174 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d72866ac24e40a38b58902f9a0184a2)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:16.972405 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769836dc-d127-4c31-bd32-db169502a174 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:17.016022 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:17.034545 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-769836dc-d127-4c31-bd32-db169502a174 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d72866ac24e40a38b58902f9a0184a2'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:17.046906 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-769836dc-d127-4c31-bd32-db169502a174 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d72866ac24e40a38b58902f9a0184a2'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:17.047654 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-769836dc-d127-4c31-bd32-db169502a174 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d72866ac24e40a38b58902f9a0184a2'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:17.048345 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-769836dc-d127-4c31-bd32-db169502a174 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d72866ac24e40a38b58902f9a0184a2'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:17.055861 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-769836dc-d127-4c31-bd32-db169502a174 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d72866ac24e40a38b58902f9a0184a2'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:17.071457 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1228/2457] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:26:16 2022] DELETE /identity/v3/users/6d72866ac24e40a38b58902f9a0184a2 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:26:17.076318 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57f481-41c1-4e7d-a3df-0a4245e2919c None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:17.076840 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57f481-41c1-4e7d-a3df-0a4245e2919c None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:17.076840 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57f481-41c1-4e7d-a3df-0a4245e2919c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:17.094599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-1f57f481-41c1-4e7d-a3df-0a4245e2919c None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:26:17.095917 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-1f57f481-41c1-4e7d-a3df-0a4245e2919c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:17.177841 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1230/2458] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:26:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:26:17.189507 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769836dc-d127-4c31-bd32-db169502a174 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:17.191153 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-769836dc-d127-4c31-bd32-db169502a174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:17.249750 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769836dc-d127-4c31-bd32-db169502a174 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:17.254212 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f2e36-c0e9-40cd-a17c-07a918344358 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:17.254862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f2e36-c0e9-40cd-a17c-07a918344358 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:17.255164 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f2e36-c0e9-40cd-a17c-07a918344358 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:17.255426 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f2e36-c0e9-40cd-a17c-07a918344358 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:17.264615 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f2e36-c0e9-40cd-a17c-07a918344358 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:17.266035 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f2e36-c0e9-40cd-a17c-07a918344358 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:17.276324 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1229/2459] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:26:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:26:17.563264 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57f481-41c1-4e7d-a3df-0a4245e2919c None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:17.565090 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-1f57f481-41c1-4e7d-a3df-0a4245e2919c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:17.579780 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57f481-41c1-4e7d-a3df-0a4245e2919c None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:17.585486 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a967e8-0f1e-4acc-b290-c6f0cb8f22b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:17.586301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a967e8-0f1e-4acc-b290-c6f0cb8f22b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:17.586581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a967e8-0f1e-4acc-b290-c6f0cb8f22b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:17.586841 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a967e8-0f1e-4acc-b290-c6f0cb8f22b8 admin admin] PATH_INFO: `/v3/projects/7b99ef7e6b8d43e2a06661021656d400` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:17.588084 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a967e8-0f1e-4acc-b290-c6f0cb8f22b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b99ef7e6b8d43e2a06661021656d400)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:17.590000 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a967e8-0f1e-4acc-b290-c6f0cb8f22b8 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:17.640227 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-53a967e8-0f1e-4acc-b290-c6f0cb8f22b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b99ef7e6b8d43e2a06661021656d400', 'request_id': 'req-53a967e8-0f1e-4acc-b290-c6f0cb8f22b8'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:17.645897 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-53a967e8-0f1e-4acc-b290-c6f0cb8f22b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b99ef7e6b8d43e2a06661021656d400', 'request_id': 'req-53a967e8-0f1e-4acc-b290-c6f0cb8f22b8'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:17.654031 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-53a967e8-0f1e-4acc-b290-c6f0cb8f22b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b99ef7e6b8d43e2a06661021656d400', 'request_id': 'req-53a967e8-0f1e-4acc-b290-c6f0cb8f22b8'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:17.665567 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-53a967e8-0f1e-4acc-b290-c6f0cb8f22b8 admin admin] The token cache is being invalidate because project 7b99ef7e6b8d43e2a06661021656d400 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:26:17.665866 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-53a967e8-0f1e-4acc-b290-c6f0cb8f22b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:17.666551 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-53a967e8-0f1e-4acc-b290-c6f0cb8f22b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:17.668334 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1231/2460] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:26:17 2022] DELETE /identity/v3/projects/7b99ef7e6b8d43e2a06661021656d400 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:26:17.672261 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f2e36-c0e9-40cd-a17c-07a918344358 service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:17.685639 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f2e36-c0e9-40cd-a17c-07a918344358 service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:17.745372 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9281150e-7b61-40ea-a97b-bb9aea1b0687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:17.746296 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9281150e-7b61-40ea-a97b-bb9aea1b0687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:17.746577 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9281150e-7b61-40ea-a97b-bb9aea1b0687 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:17.746855 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9281150e-7b61-40ea-a97b-bb9aea1b0687 admin admin] PATH_INFO: `/v3/users/955b16ce8b3247f1b530456013fbded1` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:17.748107 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9281150e-7b61-40ea-a97b-bb9aea1b0687 admin admin] RBAC: Authorizing `identity:delete_user(user_id=955b16ce8b3247f1b530456013fbded1)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:17.750054 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9281150e-7b61-40ea-a97b-bb9aea1b0687 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:17.807240 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-9281150e-7b61-40ea-a97b-bb9aea1b0687 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '955b16ce8b3247f1b530456013fbded1'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:17.812276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:17.821336 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-9281150e-7b61-40ea-a97b-bb9aea1b0687 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '955b16ce8b3247f1b530456013fbded1'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:17.822209 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-9281150e-7b61-40ea-a97b-bb9aea1b0687 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '955b16ce8b3247f1b530456013fbded1'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:17.822858 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-9281150e-7b61-40ea-a97b-bb9aea1b0687 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '955b16ce8b3247f1b530456013fbded1'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:17.829729 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-9281150e-7b61-40ea-a97b-bb9aea1b0687 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '955b16ce8b3247f1b530456013fbded1'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:17.843754 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1230/2461] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:26:17 2022] DELETE /identity/v3/users/955b16ce8b3247f1b530456013fbded1 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:26:18.138512 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a967e8-0f1e-4acc-b290-c6f0cb8f22b8 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:18.140044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-53a967e8-0f1e-4acc-b290-c6f0cb8f22b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:18.177507 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a967e8-0f1e-4acc-b290-c6f0cb8f22b8 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:18.258520 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b41855a-1759-4820-8f16-b44e568cdefa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:18.259405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41855a-1759-4820-8f16-b44e568cdefa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:18.259688 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41855a-1759-4820-8f16-b44e568cdefa admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:18.259962 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41855a-1759-4820-8f16-b44e568cdefa admin admin] PATH_INFO: `/v3/projects/20e845c95ac5437aaa11182d4ed6163e` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:18.261423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b41855a-1759-4820-8f16-b44e568cdefa admin admin] RBAC: Authorizing `identity:delete_project(project_id=20e845c95ac5437aaa11182d4ed6163e)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:18.263384 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b41855a-1759-4820-8f16-b44e568cdefa admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:18.265013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:18.312795 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-5b41855a-1759-4820-8f16-b44e568cdefa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20e845c95ac5437aaa11182d4ed6163e', 'request_id': 'req-5b41855a-1759-4820-8f16-b44e568cdefa'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:18.318677 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-5b41855a-1759-4820-8f16-b44e568cdefa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20e845c95ac5437aaa11182d4ed6163e', 'request_id': 'req-5b41855a-1759-4820-8f16-b44e568cdefa'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:18.328656 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-5b41855a-1759-4820-8f16-b44e568cdefa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20e845c95ac5437aaa11182d4ed6163e', 'request_id': 'req-5b41855a-1759-4820-8f16-b44e568cdefa'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:18.340242 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-5b41855a-1759-4820-8f16-b44e568cdefa admin admin] The token cache is being invalidate because project 20e845c95ac5437aaa11182d4ed6163e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:26:18.340534 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-5b41855a-1759-4820-8f16-b44e568cdefa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:18.341195 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-5b41855a-1759-4820-8f16-b44e568cdefa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:18.342780 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1232/2462] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:26:18 2022] DELETE /identity/v3/projects/20e845c95ac5437aaa11182d4ed6163e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:26:18.346918 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9281150e-7b61-40ea-a97b-bb9aea1b0687 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:18.348415 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-9281150e-7b61-40ea-a97b-bb9aea1b0687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:18.381695 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9281150e-7b61-40ea-a97b-bb9aea1b0687 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:18.437693 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23dac9-195f-4bea-b487-2b5572510fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:18.438662 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23dac9-195f-4bea-b487-2b5572510fed admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:18.438922 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23dac9-195f-4bea-b487-2b5572510fed admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:18.439238 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23dac9-195f-4bea-b487-2b5572510fed admin admin] PATH_INFO: `/v3/users/b77e4e4bb2a1419ba6e274187b64636e` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:18.440585 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f23dac9-195f-4bea-b487-2b5572510fed admin admin] RBAC: Authorizing `identity:delete_user(user_id=b77e4e4bb2a1419ba6e274187b64636e)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:18.442495 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f23dac9-195f-4bea-b487-2b5572510fed admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:18.493034 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-0f23dac9-195f-4bea-b487-2b5572510fed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b77e4e4bb2a1419ba6e274187b64636e'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:18.505554 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-0f23dac9-195f-4bea-b487-2b5572510fed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b77e4e4bb2a1419ba6e274187b64636e'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:18.506347 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-0f23dac9-195f-4bea-b487-2b5572510fed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b77e4e4bb2a1419ba6e274187b64636e'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:18.507073 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-0f23dac9-195f-4bea-b487-2b5572510fed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b77e4e4bb2a1419ba6e274187b64636e'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:18.520684 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-0f23dac9-195f-4bea-b487-2b5572510fed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b77e4e4bb2a1419ba6e274187b64636e'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:18.544001 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1231/2463] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:26:18 2022] DELETE /identity/v3/users/b77e4e4bb2a1419ba6e274187b64636e => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:26:18.831986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b41855a-1759-4820-8f16-b44e568cdefa admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:18.833791 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-5b41855a-1759-4820-8f16-b44e568cdefa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:18.846824 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b41855a-1759-4820-8f16-b44e568cdefa admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:18.904615 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e147ab-ea94-4de2-b76e-2b68a0dc0a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:18.905518 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e147ab-ea94-4de2-b76e-2b68a0dc0a5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:18.905803 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e147ab-ea94-4de2-b76e-2b68a0dc0a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:18.906103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e147ab-ea94-4de2-b76e-2b68a0dc0a5f admin admin] PATH_INFO: `/v3/projects/fe2ad1bc4cd44516a59bfbebee205840` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:18.907409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e147ab-ea94-4de2-b76e-2b68a0dc0a5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe2ad1bc4cd44516a59bfbebee205840)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:18.909391 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e147ab-ea94-4de2-b76e-2b68a0dc0a5f admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:18.965820 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-37e147ab-ea94-4de2-b76e-2b68a0dc0a5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe2ad1bc4cd44516a59bfbebee205840', 'request_id': 'req-37e147ab-ea94-4de2-b76e-2b68a0dc0a5f'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:18.971975 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-37e147ab-ea94-4de2-b76e-2b68a0dc0a5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe2ad1bc4cd44516a59bfbebee205840', 'request_id': 'req-37e147ab-ea94-4de2-b76e-2b68a0dc0a5f'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:18.981530 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-37e147ab-ea94-4de2-b76e-2b68a0dc0a5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe2ad1bc4cd44516a59bfbebee205840', 'request_id': 'req-37e147ab-ea94-4de2-b76e-2b68a0dc0a5f'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:18.993376 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-37e147ab-ea94-4de2-b76e-2b68a0dc0a5f admin admin] The token cache is being invalidate because project fe2ad1bc4cd44516a59bfbebee205840 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:26:18.993666 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-37e147ab-ea94-4de2-b76e-2b68a0dc0a5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:18.994368 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-37e147ab-ea94-4de2-b76e-2b68a0dc0a5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:18.996130 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1233/2464] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:26:18 2022] DELETE /identity/v3/projects/fe2ad1bc4cd44516a59bfbebee205840 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:26:19.001819 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57483e22-297e-4bba-9a29-386a59f799ce None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:19.002089 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57483e22-297e-4bba-9a29-386a59f799ce None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:19.002390 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57483e22-297e-4bba-9a29-386a59f799ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:19.024849 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-57483e22-297e-4bba-9a29-386a59f799ce None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:26:19.026028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-57483e22-297e-4bba-9a29-386a59f799ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:19.052858 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:19.090647 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1232/2465] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:26:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:26:19.100344 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507614cf-a561-4155-b124-b862284abcde None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:19.100655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507614cf-a561-4155-b124-b862284abcde None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:19.100933 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507614cf-a561-4155-b124-b862284abcde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:19.122078 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-507614cf-a561-4155-b124-b862284abcde None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:26:19.123339 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-507614cf-a561-4155-b124-b862284abcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:19.132418 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1234/2466] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:26:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:26:19.136875 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57483e22-297e-4bba-9a29-386a59f799ce None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:19.147867 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57483e22-297e-4bba-9a29-386a59f799ce None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:19.153830 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bacc74-8235-4952-b1c5-a5f0ad456b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:19.154537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bacc74-8235-4952-b1c5-a5f0ad456b3c admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:19.154824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bacc74-8235-4952-b1c5-a5f0ad456b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:19.155138 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bacc74-8235-4952-b1c5-a5f0ad456b3c admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:19.155876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bacc74-8235-4952-b1c5-a5f0ad456b3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:19.156191 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bacc74-8235-4952-b1c5-a5f0ad456b3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:19.157808 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bacc74-8235-4952-b1c5-a5f0ad456b3c admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:19.170437 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1233/2467] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 15:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:26:19.174998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507614cf-a561-4155-b124-b862284abcde None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:19.184873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507614cf-a561-4155-b124-b862284abcde None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:19.190841 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41add850-cc00-4527-ad72-0a9a805d6c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:19.191531 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41add850-cc00-4527-ad72-0a9a805d6c0b admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:19.191812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41add850-cc00-4527-ad72-0a9a805d6c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:19.192074 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41add850-cc00-4527-ad72-0a9a805d6c0b admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:19.192756 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41add850-cc00-4527-ad72-0a9a805d6c0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:19.194653 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41add850-cc00-4527-ad72-0a9a805d6c0b admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:19.205059 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:19.223300 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1235/2468] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:26:19 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:26:19.227675 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bacc74-8235-4952-b1c5-a5f0ad456b3c admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:19.238178 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bacc74-8235-4952-b1c5-a5f0ad456b3c admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:19.296906 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62953dd3-4b84-42b5-9d1d-a0cfd0cab777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:19.297589 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62953dd3-4b84-42b5-9d1d-a0cfd0cab777 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:19.297877 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62953dd3-4b84-42b5-9d1d-a0cfd0cab777 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:19.298164 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62953dd3-4b84-42b5-9d1d-a0cfd0cab777 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:19.298858 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62953dd3-4b84-42b5-9d1d-a0cfd0cab777 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:19.300817 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62953dd3-4b84-42b5-9d1d-a0cfd0cab777 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:19.323357 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1234/2469] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:26:19 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:26:19.327656 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41add850-cc00-4527-ad72-0a9a805d6c0b admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:19.338129 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41add850-cc00-4527-ad72-0a9a805d6c0b admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:19.343089 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7db330-b6cc-476c-9c92-79056bf16fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:19.343846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7db330-b6cc-476c-9c92-79056bf16fbe admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:19.344133 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7db330-b6cc-476c-9c92-79056bf16fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:19.344405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7db330-b6cc-476c-9c92-79056bf16fbe admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:19.345263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7db330-b6cc-476c-9c92-79056bf16fbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:19.346857 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7db330-b6cc-476c-9c92-79056bf16fbe admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:19.357489 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1236/2470] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:26:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:26:19.361695 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62953dd3-4b84-42b5-9d1d-a0cfd0cab777 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:19.372025 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62953dd3-4b84-42b5-9d1d-a0cfd0cab777 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:19.377451 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265cef1b-ad30-4eae-9007-872ac2fd447f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:19.378118 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265cef1b-ad30-4eae-9007-872ac2fd447f admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:19.378415 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265cef1b-ad30-4eae-9007-872ac2fd447f admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:19.378680 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265cef1b-ad30-4eae-9007-872ac2fd447f admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:19.379448 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265cef1b-ad30-4eae-9007-872ac2fd447f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:19.381057 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265cef1b-ad30-4eae-9007-872ac2fd447f admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:19.391894 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1235/2471] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:26:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:26:19.396281 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7db330-b6cc-476c-9c92-79056bf16fbe admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:19.407221 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7db330-b6cc-476c-9c92-79056bf16fbe admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:19.413042 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c357db-ed25-476a-bee0-10692df17526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:19.413954 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c357db-ed25-476a-bee0-10692df17526 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:19.414269 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c357db-ed25-476a-bee0-10692df17526 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:19.414552 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c357db-ed25-476a-bee0-10692df17526 admin admin] PATH_INFO: `/v3/projects/e8559fa92dc847c3be06878375999c42/users/f2063d7f55074113abb0d34e6f812227/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:19.430476 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c357db-ed25-476a-bee0-10692df17526 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8559fa92dc847c3be06878375999c42, user_id=f2063d7f55074113abb0d34e6f812227, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:19.433163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c357db-ed25-476a-bee0-10692df17526 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:19.447557 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1237/2472] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:26:19 2022] PUT /identity/v3/projects/e8559fa92dc847c3be06878375999c42/users/f2063d7f55074113abb0d34e6f812227/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:26:19.458151 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e8f70-28f2-4059-b5d0-da74d87f7b3a None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:19.458444 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e8f70-28f2-4059-b5d0-da74d87f7b3a None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:19.458725 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e8f70-28f2-4059-b5d0-da74d87f7b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:19.487926 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-093e8f70-28f2-4059-b5d0-da74d87f7b3a None None] MFA Rules not processed for user `f2063d7f55074113abb0d34e6f812227`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:26:19.488649 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-093e8f70-28f2-4059-b5d0-da74d87f7b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:19.555341 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1236/2473] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:26:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:26:19.566189 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c357db-ed25-476a-bee0-10692df17526 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:19.568105 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-b3c357db-ed25-476a-bee0-10692df17526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:19.635995 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c357db-ed25-476a-bee0-10692df17526 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:19.642769 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9447523-1f5f-433c-8b73-e851eb982bbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:19.643496 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9447523-1f5f-433c-8b73-e851eb982bbb service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:19.643821 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9447523-1f5f-433c-8b73-e851eb982bbb service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:19.644095 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9447523-1f5f-433c-8b73-e851eb982bbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:19.653716 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9447523-1f5f-433c-8b73-e851eb982bbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:19.655334 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9447523-1f5f-433c-8b73-e851eb982bbb service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:19.664737 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1238/2474] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:26:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:26:21.058936 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e8f70-28f2-4059-b5d0-da74d87f7b3a None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:21.060623 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-093e8f70-28f2-4059-b5d0-da74d87f7b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:21.095547 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:21.126026 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e8f70-28f2-4059-b5d0-da74d87f7b3a None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:21.130172 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd7e0c-c3f4-4e73-b985-8ebc4e67aa54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:21.130834 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd7e0c-c3f4-4e73-b985-8ebc4e67aa54 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:21.131151 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd7e0c-c3f4-4e73-b985-8ebc4e67aa54 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:21.131432 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd7e0c-c3f4-4e73-b985-8ebc4e67aa54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:21.133530 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-effd7e0c-c3f4-4e73-b985-8ebc4e67aa54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:21.200581 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effd7e0c-c3f4-4e73-b985-8ebc4e67aa54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:21.202366 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effd7e0c-c3f4-4e73-b985-8ebc4e67aa54 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:21.212183 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1237/2475] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:26:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:26:21.738676 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9447523-1f5f-433c-8b73-e851eb982bbb service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:21.740528 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-e9447523-1f5f-433c-8b73-e851eb982bbb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:21.755126 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9447523-1f5f-433c-8b73-e851eb982bbb service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:21.759925 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f95f8-e452-44af-a3f0-4dc25a0e8f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:21.760807 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f95f8-e452-44af-a3f0-4dc25a0e8f92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:21.761090 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f95f8-e452-44af-a3f0-4dc25a0e8f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:21.761365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f95f8-e452-44af-a3f0-4dc25a0e8f92 admin admin] PATH_INFO: `/v3/users/bc2f1a0d431341d6a5d1a8c5de131ece` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:21.762605 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f95f8-e452-44af-a3f0-4dc25a0e8f92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc2f1a0d431341d6a5d1a8c5de131ece)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:21.764556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f95f8-e452-44af-a3f0-4dc25a0e8f92 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:21.814495 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-2b4f95f8-e452-44af-a3f0-4dc25a0e8f92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc2f1a0d431341d6a5d1a8c5de131ece'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:21.826527 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-2b4f95f8-e452-44af-a3f0-4dc25a0e8f92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc2f1a0d431341d6a5d1a8c5de131ece'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:21.827226 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-2b4f95f8-e452-44af-a3f0-4dc25a0e8f92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc2f1a0d431341d6a5d1a8c5de131ece'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:21.827829 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-2b4f95f8-e452-44af-a3f0-4dc25a0e8f92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc2f1a0d431341d6a5d1a8c5de131ece'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:21.834830 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-2b4f95f8-e452-44af-a3f0-4dc25a0e8f92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc2f1a0d431341d6a5d1a8c5de131ece'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:21.849575 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1239/2476] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:26:21 2022] DELETE /identity/v3/users/bc2f1a0d431341d6a5d1a8c5de131ece => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:26:22.042269 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd7e0c-c3f4-4e73-b985-8ebc4e67aa54 service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:22.052894 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd7e0c-c3f4-4e73-b985-8ebc4e67aa54 service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:22.111526 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3203abc8-66ec-4e72-a8c4-03d8eb78906e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:22.112480 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203abc8-66ec-4e72-a8c4-03d8eb78906e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:22.112767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203abc8-66ec-4e72-a8c4-03d8eb78906e admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:22.113046 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203abc8-66ec-4e72-a8c4-03d8eb78906e admin admin] PATH_INFO: `/v3/projects/1cdab962e8b0462b8d1d28dcde703c15` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:22.114228 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3203abc8-66ec-4e72-a8c4-03d8eb78906e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cdab962e8b0462b8d1d28dcde703c15)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:22.116188 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3203abc8-66ec-4e72-a8c4-03d8eb78906e admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:22.166309 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-3203abc8-66ec-4e72-a8c4-03d8eb78906e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cdab962e8b0462b8d1d28dcde703c15', 'request_id': 'req-3203abc8-66ec-4e72-a8c4-03d8eb78906e'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:22.174393 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-3203abc8-66ec-4e72-a8c4-03d8eb78906e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cdab962e8b0462b8d1d28dcde703c15', 'request_id': 'req-3203abc8-66ec-4e72-a8c4-03d8eb78906e'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:22.181052 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-3203abc8-66ec-4e72-a8c4-03d8eb78906e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cdab962e8b0462b8d1d28dcde703c15', 'request_id': 'req-3203abc8-66ec-4e72-a8c4-03d8eb78906e'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:22.195588 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-3203abc8-66ec-4e72-a8c4-03d8eb78906e admin admin] The token cache is being invalidate because project 1cdab962e8b0462b8d1d28dcde703c15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:26:22.195872 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-3203abc8-66ec-4e72-a8c4-03d8eb78906e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:22.196478 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-3203abc8-66ec-4e72-a8c4-03d8eb78906e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:22.198046 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1238/2477] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:26:22 2022] DELETE /identity/v3/projects/1cdab962e8b0462b8d1d28dcde703c15 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:26:27.739682 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f95f8-e452-44af-a3f0-4dc25a0e8f92 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:27.741197 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-2b4f95f8-e452-44af-a3f0-4dc25a0e8f92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:27.776733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:27.841988 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f95f8-e452-44af-a3f0-4dc25a0e8f92 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:27.910775 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e095ea97-678b-4b54-be92-bbb4cb66ff85 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb857f32bbc04d82bfa681bd5659765f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:27.911462 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095ea97-678b-4b54-be92-bbb4cb66ff85 service placement] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:27.911742 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095ea97-678b-4b54-be92-bbb4cb66ff85 service placement] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:27.912045 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095ea97-678b-4b54-be92-bbb4cb66ff85 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:27.917625 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-e095ea97-678b-4b54-be92-bbb4cb66ff85 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:27.938207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e095ea97-678b-4b54-be92-bbb4cb66ff85 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:27.941799 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e095ea97-678b-4b54-be92-bbb4cb66ff85 service placement] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:27.957669 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1240/2478] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:26:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:26:29.669042 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3203abc8-66ec-4e72-a8c4-03d8eb78906e admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:29.671265 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-3203abc8-66ec-4e72-a8c4-03d8eb78906e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:29.682236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:29.741540 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3203abc8-66ec-4e72-a8c4-03d8eb78906e admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:29.746872 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38177765-d7ab-495d-8fe9-4f9929c77a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:29.747805 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38177765-d7ab-495d-8fe9-4f9929c77a84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:29.748094 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38177765-d7ab-495d-8fe9-4f9929c77a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:29.748368 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38177765-d7ab-495d-8fe9-4f9929c77a84 admin admin] PATH_INFO: `/v3/users/f2063d7f55074113abb0d34e6f812227` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:29.749613 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38177765-d7ab-495d-8fe9-4f9929c77a84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2063d7f55074113abb0d34e6f812227)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:29.751661 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38177765-d7ab-495d-8fe9-4f9929c77a84 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:29.798890 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-38177765-d7ab-495d-8fe9-4f9929c77a84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2063d7f55074113abb0d34e6f812227'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:29.810351 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-38177765-d7ab-495d-8fe9-4f9929c77a84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2063d7f55074113abb0d34e6f812227'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:29.811070 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-38177765-d7ab-495d-8fe9-4f9929c77a84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2063d7f55074113abb0d34e6f812227'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:29.811720 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-38177765-d7ab-495d-8fe9-4f9929c77a84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2063d7f55074113abb0d34e6f812227'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:29.819112 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-38177765-d7ab-495d-8fe9-4f9929c77a84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2063d7f55074113abb0d34e6f812227'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:29.834536 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1239/2479] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:26:29 2022] DELETE /identity/v3/users/f2063d7f55074113abb0d34e6f812227 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:26:29.839572 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be03415-52a7-4e6b-812d-5e625805b7d4 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:29.840087 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be03415-52a7-4e6b-812d-5e625805b7d4 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:29.841588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be03415-52a7-4e6b-812d-5e625805b7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:29.858795 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-2be03415-52a7-4e6b-812d-5e625805b7d4 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:26:29.861152 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-2be03415-52a7-4e6b-812d-5e625805b7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:29.931195 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1241/2480] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:26:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:26:29.941287 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38177765-d7ab-495d-8fe9-4f9929c77a84 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:29.942948 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-38177765-d7ab-495d-8fe9-4f9929c77a84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:30.013327 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38177765-d7ab-495d-8fe9-4f9929c77a84 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:30.017391 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30daeeb-7500-4e01-87c3-1141f2c0c390 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:30.018049 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30daeeb-7500-4e01-87c3-1141f2c0c390 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:30.018309 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30daeeb-7500-4e01-87c3-1141f2c0c390 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:30.018616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30daeeb-7500-4e01-87c3-1141f2c0c390 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:30.031539 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30daeeb-7500-4e01-87c3-1141f2c0c390 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:30.033237 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30daeeb-7500-4e01-87c3-1141f2c0c390 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:30.043656 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1240/2481] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:26:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:26:30.346210 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be03415-52a7-4e6b-812d-5e625805b7d4 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:30.347863 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-2be03415-52a7-4e6b-812d-5e625805b7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:30.360788 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be03415-52a7-4e6b-812d-5e625805b7d4 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:30.365879 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b429081-2c16-49df-bf1d-50aa5fdb0b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:30.366762 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b429081-2c16-49df-bf1d-50aa5fdb0b0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:30.367070 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b429081-2c16-49df-bf1d-50aa5fdb0b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:30.367329 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b429081-2c16-49df-bf1d-50aa5fdb0b0d admin admin] PATH_INFO: `/v3/projects/e8559fa92dc847c3be06878375999c42` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:30.369379 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b429081-2c16-49df-bf1d-50aa5fdb0b0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8559fa92dc847c3be06878375999c42)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:30.371902 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b429081-2c16-49df-bf1d-50aa5fdb0b0d admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:30.413246 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:30.421209 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-0b429081-2c16-49df-bf1d-50aa5fdb0b0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8559fa92dc847c3be06878375999c42', 'request_id': 'req-0b429081-2c16-49df-bf1d-50aa5fdb0b0d'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:30.426868 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-0b429081-2c16-49df-bf1d-50aa5fdb0b0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8559fa92dc847c3be06878375999c42', 'request_id': 'req-0b429081-2c16-49df-bf1d-50aa5fdb0b0d'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:30.435939 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-0b429081-2c16-49df-bf1d-50aa5fdb0b0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8559fa92dc847c3be06878375999c42', 'request_id': 'req-0b429081-2c16-49df-bf1d-50aa5fdb0b0d'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:30.447034 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-0b429081-2c16-49df-bf1d-50aa5fdb0b0d admin admin] The token cache is being invalidate because project e8559fa92dc847c3be06878375999c42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:26:30.447354 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-0b429081-2c16-49df-bf1d-50aa5fdb0b0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:30.447985 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-0b429081-2c16-49df-bf1d-50aa5fdb0b0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:26:30.449538 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1242/2482] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:26:30 2022] DELETE /identity/v3/projects/e8559fa92dc847c3be06878375999c42 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:26:30.456718 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3affd5a-4360-4538-970d-28f118c05bb8 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:30.457010 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3affd5a-4360-4538-970d-28f118c05bb8 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:30.457286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3affd5a-4360-4538-970d-28f118c05bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:30.479668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-f3affd5a-4360-4538-970d-28f118c05bb8 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:26:30.480876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-f3affd5a-4360-4538-970d-28f118c05bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:30.507820 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:30.544068 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1241/2483] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:26:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:26:30.552891 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afef079-fe11-4e27-886f-3d4e72806751 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:30.553207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afef079-fe11-4e27-886f-3d4e72806751 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:30.553502 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afef079-fe11-4e27-886f-3d4e72806751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:30.573645 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-8afef079-fe11-4e27-886f-3d4e72806751 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:26:30.574933 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-8afef079-fe11-4e27-886f-3d4e72806751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:30.584511 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1243/2484] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:26:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:26:30.590416 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3affd5a-4360-4538-970d-28f118c05bb8 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:30.602348 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3affd5a-4360-4538-970d-28f118c05bb8 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:30.608214 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93780ca5-3868-477d-aed7-2fc62e4052ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:30.608902 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93780ca5-3868-477d-aed7-2fc62e4052ac admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:30.609515 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93780ca5-3868-477d-aed7-2fc62e4052ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:30.609752 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93780ca5-3868-477d-aed7-2fc62e4052ac admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:30.610487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93780ca5-3868-477d-aed7-2fc62e4052ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:30.610797 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93780ca5-3868-477d-aed7-2fc62e4052ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:30.612467 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93780ca5-3868-477d-aed7-2fc62e4052ac admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:30.624568 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1242/2485] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 15:26:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:26:30.629162 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afef079-fe11-4e27-886f-3d4e72806751 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:30.638625 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afef079-fe11-4e27-886f-3d4e72806751 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:30.643974 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917ecd8-f072-4bcb-beaf-9dd6e782ce0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:30.644626 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917ecd8-f072-4bcb-beaf-9dd6e782ce0f admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:30.644909 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917ecd8-f072-4bcb-beaf-9dd6e782ce0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:30.645183 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917ecd8-f072-4bcb-beaf-9dd6e782ce0f admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:30.645872 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a917ecd8-f072-4bcb-beaf-9dd6e782ce0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:30.647807 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a917ecd8-f072-4bcb-beaf-9dd6e782ce0f admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:30.679327 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1244/2486] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:26:30 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:26:30.684141 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93780ca5-3868-477d-aed7-2fc62e4052ac admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:30.695687 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93780ca5-3868-477d-aed7-2fc62e4052ac admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:30.761954 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902ee0b-7d63-4bd0-98cf-02c994ed9539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:30.762630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902ee0b-7d63-4bd0-98cf-02c994ed9539 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:30.762917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902ee0b-7d63-4bd0-98cf-02c994ed9539 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:30.763232 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902ee0b-7d63-4bd0-98cf-02c994ed9539 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:30.763936 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902ee0b-7d63-4bd0-98cf-02c994ed9539 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:30.765894 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902ee0b-7d63-4bd0-98cf-02c994ed9539 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:30.788829 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1243/2487] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:26:30 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:26:30.793270 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917ecd8-f072-4bcb-beaf-9dd6e782ce0f admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:30.805015 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917ecd8-f072-4bcb-beaf-9dd6e782ce0f admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:30.810363 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30568eb2-fda9-4f84-8851-1b5ca25b1452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:30.811036 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30568eb2-fda9-4f84-8851-1b5ca25b1452 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:30.811356 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30568eb2-fda9-4f84-8851-1b5ca25b1452 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:30.811642 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30568eb2-fda9-4f84-8851-1b5ca25b1452 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:30.812384 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30568eb2-fda9-4f84-8851-1b5ca25b1452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:30.814105 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30568eb2-fda9-4f84-8851-1b5ca25b1452 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:30.824865 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1245/2488] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:26:30.828997 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902ee0b-7d63-4bd0-98cf-02c994ed9539 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:30.839162 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902ee0b-7d63-4bd0-98cf-02c994ed9539 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:30.843958 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a4c4f-b16b-4ae1-8bb2-5a746d45ab43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:30.844640 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a4c4f-b16b-4ae1-8bb2-5a746d45ab43 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:30.844922 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a4c4f-b16b-4ae1-8bb2-5a746d45ab43 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:30.845199 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a4c4f-b16b-4ae1-8bb2-5a746d45ab43 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:30.845957 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a4c4f-b16b-4ae1-8bb2-5a746d45ab43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:30.847684 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a4c4f-b16b-4ae1-8bb2-5a746d45ab43 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:30.857558 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1244/2489] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:26:30.862065 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30568eb2-fda9-4f84-8851-1b5ca25b1452 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:30.872463 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30568eb2-fda9-4f84-8851-1b5ca25b1452 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:30.882397 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89335d3b-04b8-4b36-b266-bc4aa9600446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:30.883606 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89335d3b-04b8-4b36-b266-bc4aa9600446 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:30.883968 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89335d3b-04b8-4b36-b266-bc4aa9600446 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:30.884324 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89335d3b-04b8-4b36-b266-bc4aa9600446 admin admin] PATH_INFO: `/v3/projects/535d117f782341d5aa579a8fb34c35c5/users/9b70cbf2ad4f405b84cbea240b83cd05/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:30.903628 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89335d3b-04b8-4b36-b266-bc4aa9600446 admin admin] RBAC: Authorizing `identity:create_grant(project_id=535d117f782341d5aa579a8fb34c35c5, user_id=9b70cbf2ad4f405b84cbea240b83cd05, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:30.906319 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89335d3b-04b8-4b36-b266-bc4aa9600446 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:30.922987 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1246/2490] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:26:30 2022] PUT /identity/v3/projects/535d117f782341d5aa579a8fb34c35c5/users/9b70cbf2ad4f405b84cbea240b83cd05/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:26:30.928315 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe99ff8-30e7-4af9-ad35-b0d99f106e02 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:30.928613 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe99ff8-30e7-4af9-ad35-b0d99f106e02 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:30.928896 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe99ff8-30e7-4af9-ad35-b0d99f106e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:30.947534 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-2fe99ff8-30e7-4af9-ad35-b0d99f106e02 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:26:30.948754 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-2fe99ff8-30e7-4af9-ad35-b0d99f106e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:31.015012 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1245/2491] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:26:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:26:31.025020 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89335d3b-04b8-4b36-b266-bc4aa9600446 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:31.026918 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-89335d3b-04b8-4b36-b266-bc4aa9600446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:31.090521 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89335d3b-04b8-4b36-b266-bc4aa9600446 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:31.097137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410dc0b-bbd7-4cca-ab96-12e4ea6e4f04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:31.098017 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410dc0b-bbd7-4cca-ab96-12e4ea6e4f04 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:31.098435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410dc0b-bbd7-4cca-ab96-12e4ea6e4f04 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:31.098755 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410dc0b-bbd7-4cca-ab96-12e4ea6e4f04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:31.114081 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6410dc0b-bbd7-4cca-ab96-12e4ea6e4f04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:31.115626 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6410dc0b-bbd7-4cca-ab96-12e4ea6e4f04 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:31.125920 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1247/2492] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:26:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:26:35.170612 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51e9da-a1b2-4247-b05c-c844bbcc588c None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:35.171176 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51e9da-a1b2-4247-b05c-c844bbcc588c None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:35.171308 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51e9da-a1b2-4247-b05c-c844bbcc588c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:35.200440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-ca51e9da-a1b2-4247-b05c-c844bbcc588c None None] MFA Rules not processed for user `9b70cbf2ad4f405b84cbea240b83cd05`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:26:35.201763 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-ca51e9da-a1b2-4247-b05c-c844bbcc588c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:35.228545 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:35.274825 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1246/2493] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:26:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:26:35.279280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410dc0b-bbd7-4cca-ab96-12e4ea6e4f04 service neutron] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:35.289538 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410dc0b-bbd7-4cca-ab96-12e4ea6e4f04 service neutron] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:35.294289 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec2b04-289e-495c-9e3d-f3175c90efe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:35.295029 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec2b04-289e-495c-9e3d-f3175c90efe6 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:35.295435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec2b04-289e-495c-9e3d-f3175c90efe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:35.295777 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec2b04-289e-495c-9e3d-f3175c90efe6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:35.296603 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dec2b04-289e-495c-9e3d-f3175c90efe6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:35.298684 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dec2b04-289e-495c-9e3d-f3175c90efe6 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:35.327742 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1248/2494] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:26:35 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:26:35.333026 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51e9da-a1b2-4247-b05c-c844bbcc588c None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:35.344716 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51e9da-a1b2-4247-b05c-c844bbcc588c None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:35.403793 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4005f2ca-6629-4326-86e3-9ec61d20d50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:35.404481 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005f2ca-6629-4326-86e3-9ec61d20d50e admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:35.404772 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005f2ca-6629-4326-86e3-9ec61d20d50e admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:35.405050 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005f2ca-6629-4326-86e3-9ec61d20d50e admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:35.405763 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4005f2ca-6629-4326-86e3-9ec61d20d50e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:35.408275 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4005f2ca-6629-4326-86e3-9ec61d20d50e admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:35.433279 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1247/2495] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:26:35 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:26:35.434826 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec2b04-289e-495c-9e3d-f3175c90efe6 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:35.446435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec2b04-289e-495c-9e3d-f3175c90efe6 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:35.451281 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7742016-f826-445a-9281-f8e75ee9089a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:35.452008 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7742016-f826-445a-9281-f8e75ee9089a admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:35.452322 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7742016-f826-445a-9281-f8e75ee9089a admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:35.452644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7742016-f826-445a-9281-f8e75ee9089a admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:35.453467 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7742016-f826-445a-9281-f8e75ee9089a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:35.455196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7742016-f826-445a-9281-f8e75ee9089a admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:35.467183 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1249/2496] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:26:35.471789 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4005f2ca-6629-4326-86e3-9ec61d20d50e admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:35.488744 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4005f2ca-6629-4326-86e3-9ec61d20d50e admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:35.495560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a4244-b0a0-475e-98aa-0c5b27ed5a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:35.496735 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a4244-b0a0-475e-98aa-0c5b27ed5a88 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:35.497254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a4244-b0a0-475e-98aa-0c5b27ed5a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:35.497711 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a4244-b0a0-475e-98aa-0c5b27ed5a88 admin admin] PATH_INFO: `/v3/projects/84297365b01d4761a82314a1aff95c38/users/2ef9fd651da04b51bef1ad1af365fba5/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:35.525548 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a4244-b0a0-475e-98aa-0c5b27ed5a88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84297365b01d4761a82314a1aff95c38, user_id=2ef9fd651da04b51bef1ad1af365fba5, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:35.529173 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a4244-b0a0-475e-98aa-0c5b27ed5a88 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:35.546080 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1248/2497] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:26:35 2022] PUT /identity/v3/projects/84297365b01d4761a82314a1aff95c38/users/2ef9fd651da04b51bef1ad1af365fba5/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:26:35.551413 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7742016-f826-445a-9281-f8e75ee9089a admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:35.562853 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7742016-f826-445a-9281-f8e75ee9089a admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:35.593646 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:35.629136 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047147b3-3838-4c11-ac07-5ddb2be663f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:35.629854 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047147b3-3838-4c11-ac07-5ddb2be663f9 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:35.630142 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047147b3-3838-4c11-ac07-5ddb2be663f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:35.631143 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047147b3-3838-4c11-ac07-5ddb2be663f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:35.631302 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047147b3-3838-4c11-ac07-5ddb2be663f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:35.634403 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047147b3-3838-4c11-ac07-5ddb2be663f9 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:35.647101 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1250/2498] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:26:35.650252 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a4244-b0a0-475e-98aa-0c5b27ed5a88 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:35.661484 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a4244-b0a0-475e-98aa-0c5b27ed5a88 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:35.666703 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de2145-52bf-499d-8964-2d1bf0e41531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:35.667796 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de2145-52bf-499d-8964-2d1bf0e41531 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:35.668083 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de2145-52bf-499d-8964-2d1bf0e41531 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:35.668361 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de2145-52bf-499d-8964-2d1bf0e41531 admin admin] PATH_INFO: `/v3/projects/84297365b01d4761a82314a1aff95c38/users/2ef9fd651da04b51bef1ad1af365fba5/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:35.670398 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de2145-52bf-499d-8964-2d1bf0e41531 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84297365b01d4761a82314a1aff95c38, user_id=2ef9fd651da04b51bef1ad1af365fba5, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:35.676039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de2145-52bf-499d-8964-2d1bf0e41531 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:35.693528 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1249/2499] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:26:35 2022] PUT /identity/v3/projects/84297365b01d4761a82314a1aff95c38/users/2ef9fd651da04b51bef1ad1af365fba5/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:26:40.144188 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b649f77-8b00-4bad-abea-caf16dc3a354 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:40.144677 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b649f77-8b00-4bad-abea-caf16dc3a354 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:40.144770 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b649f77-8b00-4bad-abea-caf16dc3a354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:40.206405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-7b649f77-8b00-4bad-abea-caf16dc3a354 None None] MFA Rules not processed for user `2ef9fd651da04b51bef1ad1af365fba5`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:26:40.207996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-7b649f77-8b00-4bad-abea-caf16dc3a354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:40.310761 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1251/2500] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:26:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:26:40.325507 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de2145-52bf-499d-8964-2d1bf0e41531 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:40.328859 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-84de2145-52bf-499d-8964-2d1bf0e41531 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:40.432666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de2145-52bf-499d-8964-2d1bf0e41531 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:40.436715 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9adafd2-8c2d-4172-af6c-da4e34e19f36 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d7d59080cd4832a45dbb6c0349cf4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:40.437399 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9adafd2-8c2d-4172-af6c-da4e34e19f36 service glance] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:40.437947 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9adafd2-8c2d-4172-af6c-da4e34e19f36 service glance] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:40.438228 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9adafd2-8c2d-4172-af6c-da4e34e19f36 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:40.462062 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:40.530468 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9adafd2-8c2d-4172-af6c-da4e34e19f36 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:40.532213 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9adafd2-8c2d-4172-af6c-da4e34e19f36 service glance] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:40.543132 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1250/2501] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:26:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:26:40.665747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31967a9-912c-4930-b296-dcdc244c5e21 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:40.666049 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31967a9-912c-4930-b296-dcdc244c5e21 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:40.666387 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31967a9-912c-4930-b296-dcdc244c5e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:40.731758 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-f31967a9-912c-4930-b296-dcdc244c5e21 None None] MFA Rules not processed for user `2f8cf6d908314281a3ddc9d9513d39c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:26:40.732896 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-f31967a9-912c-4930-b296-dcdc244c5e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:40.813083 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1252/2502] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Jul 20 15:26:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:26:40.861258 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9adafd2-8c2d-4172-af6c-da4e34e19f36 service glance] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:40.862729 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e9adafd2-8c2d-4172-af6c-da4e34e19f36 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:40.960243 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9adafd2-8c2d-4172-af6c-da4e34e19f36 service glance] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:40.965474 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa26743-6189-42e1-b6c2-0628fc555249 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f4844988474d7a91133d77906611a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:40.966138 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa26743-6189-42e1-b6c2-0628fc555249 service swift] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:40.966446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa26743-6189-42e1-b6c2-0628fc555249 service swift] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:40.966689 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa26743-6189-42e1-b6c2-0628fc555249 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:40.975881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa26743-6189-42e1-b6c2-0628fc555249 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:40.977473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa26743-6189-42e1-b6c2-0628fc555249 service swift] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:40.985999 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1251/2503] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jul 20 15:26:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:26:49.221160 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418dc9f9-5695-4441-966f-18e4a7114f22 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:49.221684 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418dc9f9-5695-4441-966f-18e4a7114f22 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:49.221811 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418dc9f9-5695-4441-966f-18e4a7114f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:49.242527 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-418dc9f9-5695-4441-966f-18e4a7114f22 None None] MFA Rules not processed for user `9b70cbf2ad4f405b84cbea240b83cd05`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:26:49.244044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-418dc9f9-5695-4441-966f-18e4a7114f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:49.252641 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1253/2504] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:26:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:26:49.287362 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa26743-6189-42e1-b6c2-0628fc555249 service swift] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:49.296757 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa26743-6189-42e1-b6c2-0628fc555249 service swift] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:49.300443 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1488a-f767-4da4-9d98-97891e9a5003 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d7d59080cd4832a45dbb6c0349cf4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:49.301088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1488a-f767-4da4-9d98-97891e9a5003 service glance] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:49.301382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1488a-f767-4da4-9d98-97891e9a5003 service glance] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:49.301683 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1488a-f767-4da4-9d98-97891e9a5003 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:49.303525 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-76e1488a-f767-4da4-9d98-97891e9a5003 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:49.385009 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=100 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:49.387495 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1488a-f767-4da4-9d98-97891e9a5003 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:49.389258 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1488a-f767-4da4-9d98-97891e9a5003 service glance] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:49.401064 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1252/2505] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:26:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:26:51.406975 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c65ca6-d73e-45b0-9e6f-b0d5d012905c None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:51.407595 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c65ca6-d73e-45b0-9e6f-b0d5d012905c None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:51.407699 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c65ca6-d73e-45b0-9e6f-b0d5d012905c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:51.426277 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-69c65ca6-d73e-45b0-9e6f-b0d5d012905c None None] MFA Rules not processed for user `2f8cf6d908314281a3ddc9d9513d39c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:26:51.427520 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-69c65ca6-d73e-45b0-9e6f-b0d5d012905c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:51.436521 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1254/2506] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Jul 20 15:26:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:26:51.449622 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1488a-f767-4da4-9d98-97891e9a5003 service glance] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:51.461442 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1488a-f767-4da4-9d98-97891e9a5003 service glance] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:51.466654 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac679a6-9ffa-4757-bdc6-fd6f1bb2e6d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f4844988474d7a91133d77906611a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:51.467334 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac679a6-9ffa-4757-bdc6-fd6f1bb2e6d2 service swift] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:51.467616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac679a6-9ffa-4757-bdc6-fd6f1bb2e6d2 service swift] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:51.467890 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac679a6-9ffa-4757-bdc6-fd6f1bb2e6d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:51.478002 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac679a6-9ffa-4757-bdc6-fd6f1bb2e6d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:51.479627 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac679a6-9ffa-4757-bdc6-fd6f1bb2e6d2 service swift] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:51.489573 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1253/2507] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jul 20 15:26:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 15:27:01.420548 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=98 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:01.486889 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=8 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:14.957101 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c65ca6-d73e-45b0-9e6f-b0d5d012905c None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:27:14.962082 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-69c65ca6-d73e-45b0-9e6f-b0d5d012905c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:27:15.014640 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c65ca6-d73e-45b0-9e6f-b0d5d012905c None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:27:15.020702 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129dcd0e-75ba-451c-af9b-ab47588b9d46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:27:15.021375 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129dcd0e-75ba-451c-af9b-ab47588b9d46 service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:15.021658 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129dcd0e-75ba-451c-af9b-ab47588b9d46 service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:15.021934 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129dcd0e-75ba-451c-af9b-ab47588b9d46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:15.023822 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-129dcd0e-75ba-451c-af9b-ab47588b9d46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:27:15.040124 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129dcd0e-75ba-451c-af9b-ab47588b9d46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:27:15.041863 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129dcd0e-75ba-451c-af9b-ab47588b9d46 service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:27:15.054182 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1255/2508] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:27:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:27:20.023141 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac679a6-9ffa-4757-bdc6-fd6f1bb2e6d2 service swift] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:27:20.043647 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac679a6-9ffa-4757-bdc6-fd6f1bb2e6d2 service swift] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:27:20.050404 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28870c3a-1ad8-48a2-8d10-9e754f33d1df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:27:20.051087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28870c3a-1ad8-48a2-8d10-9e754f33d1df service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:20.051382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28870c3a-1ad8-48a2-8d10-9e754f33d1df service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:20.051661 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28870c3a-1ad8-48a2-8d10-9e754f33d1df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:20.053685 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-28870c3a-1ad8-48a2-8d10-9e754f33d1df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:27:20.126934 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28870c3a-1ad8-48a2-8d10-9e754f33d1df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:27:20.128608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28870c3a-1ad8-48a2-8d10-9e754f33d1df service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:27:20.138460 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1254/2509] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:27:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:27:25.049046 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=17 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:29.709645 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129dcd0e-75ba-451c-af9b-ab47588b9d46 service nova] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:27:29.721442 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129dcd0e-75ba-451c-af9b-ab47588b9d46 service nova] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:27:29.727760 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24505cc9-c2e8-4b66-9782-efb64d1c03dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:27:29.728418 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24505cc9-c2e8-4b66-9782-efb64d1c03dc service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:29.728719 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24505cc9-c2e8-4b66-9782-efb64d1c03dc service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:29.728989 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24505cc9-c2e8-4b66-9782-efb64d1c03dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:29.731269 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-24505cc9-c2e8-4b66-9782-efb64d1c03dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:27:29.806643 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24505cc9-c2e8-4b66-9782-efb64d1c03dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:27:29.808309 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24505cc9-c2e8-4b66-9782-efb64d1c03dc service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:27:29.817448 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1256/2510] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:27:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:27:30.134151 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=36 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:38.997500 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28870c3a-1ad8-48a2-8d10-9e754f33d1df service nova] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:27:39.055387 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28870c3a-1ad8-48a2-8d10-9e754f33d1df service nova] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:27:39.059076 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec63b9c-87b6-4350-bd8b-af230ea7acca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:27:39.059737 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec63b9c-87b6-4350-bd8b-af230ea7acca service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:39.060027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec63b9c-87b6-4350-bd8b-af230ea7acca service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:39.060309 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec63b9c-87b6-4350-bd8b-af230ea7acca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:39.069648 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec63b9c-87b6-4350-bd8b-af230ea7acca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:27:39.071119 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec63b9c-87b6-4350-bd8b-af230ea7acca service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:27:39.080953 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1255/2511] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:27:38 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:27:39.813215 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=33 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:49.076513 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=26 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:55.621578 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676697c8-e1d0-4d88-bc70-c4a1d1d9e70a None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:55.622132 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676697c8-e1d0-4d88-bc70-c4a1d1d9e70a None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:55.622132 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676697c8-e1d0-4d88-bc70-c4a1d1d9e70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:55.640333 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-676697c8-e1d0-4d88-bc70-c4a1d1d9e70a None None] MFA Rules not processed for user `2f8cf6d908314281a3ddc9d9513d39c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:27:55.641710 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-676697c8-e1d0-4d88-bc70-c4a1d1d9e70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:27:55.649130 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1257/2512] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Jul 20 15:27:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:27:55.693362 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec63b9c-87b6-4350-bd8b-af230ea7acca service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:27:55.705440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec63b9c-87b6-4350-bd8b-af230ea7acca service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:27:55.710364 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f000cb3-427d-47e5-8b96-67836c5658d7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f4844988474d7a91133d77906611a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:27:55.711020 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f000cb3-427d-47e5-8b96-67836c5658d7 service swift] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:55.711338 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f000cb3-427d-47e5-8b96-67836c5658d7 service swift] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:55.711623 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f000cb3-427d-47e5-8b96-67836c5658d7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:55.720701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f000cb3-427d-47e5-8b96-67836c5658d7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:27:55.722299 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f000cb3-427d-47e5-8b96-67836c5658d7 service swift] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:27:55.731343 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1256/2513] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jul 20 15:27:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:28:04.073865 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33148579-e097-40c8-99a2-045a73ad27a7 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:04.074526 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33148579-e097-40c8-99a2-045a73ad27a7 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:04.074802 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33148579-e097-40c8-99a2-045a73ad27a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:04.099391 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-33148579-e097-40c8-99a2-045a73ad27a7 None None] MFA Rules not processed for user `e3574b5b8f3241b192b0b810122d4769`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:04.101107 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-33148579-e097-40c8-99a2-045a73ad27a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:04.194498 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1258/2514] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:28:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:05.727877 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=6 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:10.398932 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e327f7-10e3-4ddf-871a-09dc641ed6e0 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:10.399641 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e327f7-10e3-4ddf-871a-09dc641ed6e0 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:10.399641 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e327f7-10e3-4ddf-871a-09dc641ed6e0 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:10.401249 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1257/2515] 10.222.0.15 () {54 vars in 921 bytes} [Wed Jul 20 15:28:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:28:10.407885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6d7ce-bf01-4cbd-beac-672143c1c821 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:10.408795 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6d7ce-bf01-4cbd-beac-672143c1c821 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:10.409112 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6d7ce-bf01-4cbd-beac-672143c1c821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:10.448505 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-dee6d7ce-bf01-4cbd-beac-672143c1c821 None None] MFA Rules not processed for user `5d4a107ed04348b59d20d1703dbe04eb`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:10.449776 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-dee6d7ce-bf01-4cbd-beac-672143c1c821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:10.459337 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1259/2516] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Jul 20 15:28:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:10.470984 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e327f7-10e3-4ddf-871a-09dc641ed6e0 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:10.485321 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e327f7-10e3-4ddf-871a-09dc641ed6e0 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:10.491017 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c89ee-5514-43cb-9a00-bae0ae1cf5d4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d7d59080cd4832a45dbb6c0349cf4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:10.491739 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c89ee-5514-43cb-9a00-bae0ae1cf5d4 service glance] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:10.491994 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c89ee-5514-43cb-9a00-bae0ae1cf5d4 service glance] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:10.492439 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c89ee-5514-43cb-9a00-bae0ae1cf5d4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:10.504467 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c89ee-5514-43cb-9a00-bae0ae1cf5d4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:10.505906 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c89ee-5514-43cb-9a00-bae0ae1cf5d4 service glance] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:10.517193 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1258/2517] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:28:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:28:10.586581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b000f8-42c5-4002-910e-7228cd8f0ec0 None None] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:10.586876 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b000f8-42c5-4002-910e-7228cd8f0ec0 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:10.587228 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b000f8-42c5-4002-910e-7228cd8f0ec0 None None] PATH_INFO: `/` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:10.588614 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1260/2518] 10.222.0.15 () {54 vars in 921 bytes} [Wed Jul 20 15:28:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:28:10.594538 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344b354-f753-47ce-a65d-984bccd80f23 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:10.595106 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344b354-f753-47ce-a65d-984bccd80f23 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:10.595489 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344b354-f753-47ce-a65d-984bccd80f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:10.637855 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-3344b354-f753-47ce-a65d-984bccd80f23 None None] MFA Rules not processed for user `7658ffde740d446682c1876662815987`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:10.639214 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-3344b354-f753-47ce-a65d-984bccd80f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:10.645884 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1259/2519] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Jul 20 15:28:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:10.658822 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b000f8-42c5-4002-910e-7228cd8f0ec0 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:10.660809 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-30b000f8-42c5-4002-910e-7228cd8f0ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:10.677571 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b000f8-42c5-4002-910e-7228cd8f0ec0 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:10.681178 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcecfed-4acb-4c47-a021-7a3b2ca17f15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:10.681834 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcecfed-4acb-4c47-a021-7a3b2ca17f15 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:10.682118 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcecfed-4acb-4c47-a021-7a3b2ca17f15 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:10.682395 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcecfed-4acb-4c47-a021-7a3b2ca17f15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:10.690203 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcecfed-4acb-4c47-a021-7a3b2ca17f15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:10.691681 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcecfed-4acb-4c47-a021-7a3b2ca17f15 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:10.700235 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1261/2520] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:28:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:28:10.786783 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500b004-8568-4ada-bbff-814e842466f0 None None] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:10.787182 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500b004-8568-4ada-bbff-814e842466f0 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:10.787538 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500b004-8568-4ada-bbff-814e842466f0 None None] PATH_INFO: `/` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:10.788962 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1260/2521] 10.222.0.15 () {54 vars in 921 bytes} [Wed Jul 20 15:28:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:28:10.795000 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7a926-7359-4180-ba62-bdd39f11fb94 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:10.795328 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7a926-7359-4180-ba62-bdd39f11fb94 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:10.795607 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7a926-7359-4180-ba62-bdd39f11fb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:10.814535 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-40c7a926-7359-4180-ba62-bdd39f11fb94 None None] MFA Rules not processed for user `5d4a107ed04348b59d20d1703dbe04eb`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:10.815770 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-40c7a926-7359-4180-ba62-bdd39f11fb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:10.825269 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1262/2522] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Jul 20 15:28:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:28:10.837661 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500b004-8568-4ada-bbff-814e842466f0 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:10.855662 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500b004-8568-4ada-bbff-814e842466f0 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:10.861014 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877f0cf-ce51-48c6-83f0-5c9742a2b997 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:10.861771 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877f0cf-ce51-48c6-83f0-5c9742a2b997 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:10.862309 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877f0cf-ce51-48c6-83f0-5c9742a2b997 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:10.862813 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877f0cf-ce51-48c6-83f0-5c9742a2b997 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:10.875366 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9877f0cf-ce51-48c6-83f0-5c9742a2b997 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:10.877109 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9877f0cf-ce51-48c6-83f0-5c9742a2b997 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:10.907821 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1261/2523] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:28:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:28:10.948135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7a926-7359-4180-ba62-bdd39f11fb94 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:10.958010 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7a926-7359-4180-ba62-bdd39f11fb94 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:10.962041 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e9969-4c6d-4ca8-8fd4-d7a4673f38fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:10.963919 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e9969-4c6d-4ca8-8fd4-d7a4673f38fa service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:10.964504 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e9969-4c6d-4ca8-8fd4-d7a4673f38fa service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:10.964947 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e9969-4c6d-4ca8-8fd4-d7a4673f38fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:10.968682 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-c01e9969-4c6d-4ca8-8fd4-d7a4673f38fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:11.092055 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01e9969-4c6d-4ca8-8fd4-d7a4673f38fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:11.093485 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01e9969-4c6d-4ca8-8fd4-d7a4673f38fa service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:11.117727 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1263/2524] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:28:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:28:14.736582 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877f0cf-ce51-48c6-83f0-5c9742a2b997 service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:14.738331 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-9877f0cf-ce51-48c6-83f0-5c9742a2b997 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:14.757660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877f0cf-ce51-48c6-83f0-5c9742a2b997 service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:14.762289 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da1525-1adc-4da4-87f6-8e507e1a883a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:14.763279 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da1525-1adc-4da4-87f6-8e507e1a883a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:14.763580 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da1525-1adc-4da4-87f6-8e507e1a883a admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:14.763863 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da1525-1adc-4da4-87f6-8e507e1a883a admin admin] PATH_INFO: `/v3/users/5f51124af1e846fb89552078c81afdb0` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:14.765083 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08da1525-1adc-4da4-87f6-8e507e1a883a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f51124af1e846fb89552078c81afdb0)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:14.767057 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08da1525-1adc-4da4-87f6-8e507e1a883a admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:14.814431 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-08da1525-1adc-4da4-87f6-8e507e1a883a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f51124af1e846fb89552078c81afdb0'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:14.826410 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-08da1525-1adc-4da4-87f6-8e507e1a883a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f51124af1e846fb89552078c81afdb0'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:14.827151 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-08da1525-1adc-4da4-87f6-8e507e1a883a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f51124af1e846fb89552078c81afdb0'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:14.827778 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-08da1525-1adc-4da4-87f6-8e507e1a883a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f51124af1e846fb89552078c81afdb0'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:14.834800 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-08da1525-1adc-4da4-87f6-8e507e1a883a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f51124af1e846fb89552078c81afdb0'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:14.850440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1262/2525] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:28:14 2022] DELETE /identity/v3/users/5f51124af1e846fb89552078c81afdb0 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:15.031825 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e9969-4c6d-4ca8-8fd4-d7a4673f38fa service neutron] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:15.043340 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e9969-4c6d-4ca8-8fd4-d7a4673f38fa service neutron] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:15.064595 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:15.105214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3df5db5-fa4c-4b8a-88be-478697191ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:15.106062 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df5db5-fa4c-4b8a-88be-478697191ef8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:15.106659 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df5db5-fa4c-4b8a-88be-478697191ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:15.106927 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df5db5-fa4c-4b8a-88be-478697191ef8 admin admin] PATH_INFO: `/v3/projects/b966dbac18614c979f674666ba817def` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:15.108286 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3df5db5-fa4c-4b8a-88be-478697191ef8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b966dbac18614c979f674666ba817def)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:15.110382 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3df5db5-fa4c-4b8a-88be-478697191ef8 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:15.157883 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a3df5db5-fa4c-4b8a-88be-478697191ef8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b966dbac18614c979f674666ba817def', 'request_id': 'req-a3df5db5-fa4c-4b8a-88be-478697191ef8'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:15.163357 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a3df5db5-fa4c-4b8a-88be-478697191ef8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b966dbac18614c979f674666ba817def', 'request_id': 'req-a3df5db5-fa4c-4b8a-88be-478697191ef8'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:15.172836 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a3df5db5-fa4c-4b8a-88be-478697191ef8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b966dbac18614c979f674666ba817def', 'request_id': 'req-a3df5db5-fa4c-4b8a-88be-478697191ef8'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:15.184787 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a3df5db5-fa4c-4b8a-88be-478697191ef8 admin admin] The token cache is being invalidate because project b966dbac18614c979f674666ba817def was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:28:15.185119 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a3df5db5-fa4c-4b8a-88be-478697191ef8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:15.185844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a3df5db5-fa4c-4b8a-88be-478697191ef8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:15.187535 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1264/2526] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:28:15 2022] DELETE /identity/v3/projects/b966dbac18614c979f674666ba817def => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:28:15.192569 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d00f8-5cec-493f-8f87-52c9eaf7b3d4 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:15.192864 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d00f8-5cec-493f-8f87-52c9eaf7b3d4 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:15.193144 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d00f8-5cec-493f-8f87-52c9eaf7b3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:15.212503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-0a5d00f8-5cec-493f-8f87-52c9eaf7b3d4 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:15.213720 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-0a5d00f8-5cec-493f-8f87-52c9eaf7b3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:15.276113 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1263/2527] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:28:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:15.284339 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78e41f-596f-400f-8970-ea18da148cc1 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:15.284636 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78e41f-596f-400f-8970-ea18da148cc1 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:15.284942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78e41f-596f-400f-8970-ea18da148cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:15.304613 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-9c78e41f-596f-400f-8970-ea18da148cc1 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:15.305951 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-9c78e41f-596f-400f-8970-ea18da148cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:15.314513 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1265/2528] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:28:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:15.318884 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d00f8-5cec-493f-8f87-52c9eaf7b3d4 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:15.329999 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d00f8-5cec-493f-8f87-52c9eaf7b3d4 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:15.335515 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96072915-4362-49f1-9a0e-263350d4e1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:15.336203 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96072915-4362-49f1-9a0e-263350d4e1bc admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:15.336486 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96072915-4362-49f1-9a0e-263350d4e1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:15.336760 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96072915-4362-49f1-9a0e-263350d4e1bc admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:15.337486 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96072915-4362-49f1-9a0e-263350d4e1bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:28:15.337811 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96072915-4362-49f1-9a0e-263350d4e1bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:15.339458 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96072915-4362-49f1-9a0e-263350d4e1bc admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:15.351829 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1264/2529] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 15:28:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:28:15.356567 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78e41f-596f-400f-8970-ea18da148cc1 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:15.366745 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78e41f-596f-400f-8970-ea18da148cc1 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:15.371966 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1980e47-4497-4e60-8fd3-aa4163e0f6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:15.372656 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1980e47-4497-4e60-8fd3-aa4163e0f6f1 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:15.372946 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1980e47-4497-4e60-8fd3-aa4163e0f6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:15.373226 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1980e47-4497-4e60-8fd3-aa4163e0f6f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:15.374034 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1980e47-4497-4e60-8fd3-aa4163e0f6f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:15.376284 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1980e47-4497-4e60-8fd3-aa4163e0f6f1 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:15.407273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1266/2530] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:28:15 2022] POST /identity/v3/projects => generated 405 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:28:15.408892 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96072915-4362-49f1-9a0e-263350d4e1bc admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:15.420265 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96072915-4362-49f1-9a0e-263350d4e1bc admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:15.430178 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:15.484611 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb6bf84-ecd0-4acc-9597-7fd4592d4cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:15.485295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6bf84-ecd0-4acc-9597-7fd4592d4cd2 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:15.485577 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6bf84-ecd0-4acc-9597-7fd4592d4cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:15.485854 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6bf84-ecd0-4acc-9597-7fd4592d4cd2 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:15.486565 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb6bf84-ecd0-4acc-9597-7fd4592d4cd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:15.488561 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb6bf84-ecd0-4acc-9597-7fd4592d4cd2 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:15.513133 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1265/2531] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:28:15 2022] POST /identity/v3/users => generated 303 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:28:15.517404 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1980e47-4497-4e60-8fd3-aa4163e0f6f1 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:15.527897 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1980e47-4497-4e60-8fd3-aa4163e0f6f1 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:15.533112 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a566cc-11e6-4f06-8a5b-7bf30c715bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:15.533840 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a566cc-11e6-4f06-8a5b-7bf30c715bfc admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:15.534137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a566cc-11e6-4f06-8a5b-7bf30c715bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:15.534419 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a566cc-11e6-4f06-8a5b-7bf30c715bfc admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:15.535268 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a566cc-11e6-4f06-8a5b-7bf30c715bfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:15.536889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a566cc-11e6-4f06-8a5b-7bf30c715bfc admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:15.547516 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1267/2532] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:28:15.551578 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb6bf84-ecd0-4acc-9597-7fd4592d4cd2 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:15.562554 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb6bf84-ecd0-4acc-9597-7fd4592d4cd2 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:15.568862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c348eca-dbbf-45e3-8d7c-bf2b713f90c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:15.569511 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c348eca-dbbf-45e3-8d7c-bf2b713f90c6 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:15.569794 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c348eca-dbbf-45e3-8d7c-bf2b713f90c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:15.570073 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c348eca-dbbf-45e3-8d7c-bf2b713f90c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:15.570955 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c348eca-dbbf-45e3-8d7c-bf2b713f90c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:15.572492 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c348eca-dbbf-45e3-8d7c-bf2b713f90c6 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:15.583296 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1266/2533] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:28:15.587402 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a566cc-11e6-4f06-8a5b-7bf30c715bfc admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:15.598267 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a566cc-11e6-4f06-8a5b-7bf30c715bfc admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:15.604279 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab1bf4-48bc-4501-a3d6-4dc5f135ed9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:15.605175 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab1bf4-48bc-4501-a3d6-4dc5f135ed9f admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:15.605455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab1bf4-48bc-4501-a3d6-4dc5f135ed9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:15.605729 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab1bf4-48bc-4501-a3d6-4dc5f135ed9f admin admin] PATH_INFO: `/v3/projects/2d4203cf433b4b32b0a465e3e56111b5/users/59859d0a15914bc6bc4f063a72d9fa47/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:15.623647 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab1bf4-48bc-4501-a3d6-4dc5f135ed9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d4203cf433b4b32b0a465e3e56111b5, user_id=59859d0a15914bc6bc4f063a72d9fa47, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:15.626504 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab1bf4-48bc-4501-a3d6-4dc5f135ed9f admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:15.641253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1268/2534] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:28:15 2022] PUT /identity/v3/projects/2d4203cf433b4b32b0a465e3e56111b5/users/59859d0a15914bc6bc4f063a72d9fa47/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:15.652248 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084703b9-2dc7-46c7-898d-6955b1e730d8 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:15.652541 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084703b9-2dc7-46c7-898d-6955b1e730d8 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:15.652825 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084703b9-2dc7-46c7-898d-6955b1e730d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:15.681346 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-084703b9-2dc7-46c7-898d-6955b1e730d8 None None] MFA Rules not processed for user `59859d0a15914bc6bc4f063a72d9fa47`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:15.682566 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-084703b9-2dc7-46c7-898d-6955b1e730d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:15.747952 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1267/2535] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:28:15 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:15.752641 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab1bf4-48bc-4501-a3d6-4dc5f135ed9f admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:15.803989 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:15.815895 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab1bf4-48bc-4501-a3d6-4dc5f135ed9f admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:15.821514 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff0ef02-357f-456c-8ea2-8bf8d3f20079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:15.822173 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff0ef02-357f-456c-8ea2-8bf8d3f20079 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:15.822566 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff0ef02-357f-456c-8ea2-8bf8d3f20079 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:15.822833 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff0ef02-357f-456c-8ea2-8bf8d3f20079 admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:15.823640 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff0ef02-357f-456c-8ea2-8bf8d3f20079 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:15.825543 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff0ef02-357f-456c-8ea2-8bf8d3f20079 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:15.857315 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1269/2536] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:28:15 2022] POST /identity/v3/projects => generated 405 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:28:15.858518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084703b9-2dc7-46c7-898d-6955b1e730d8 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:15.869137 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084703b9-2dc7-46c7-898d-6955b1e730d8 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:15.927603 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80648252-3858-40f9-a636-455cd4e05bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:15.928221 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80648252-3858-40f9-a636-455cd4e05bf3 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:15.928508 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80648252-3858-40f9-a636-455cd4e05bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:15.928789 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80648252-3858-40f9-a636-455cd4e05bf3 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:15.929480 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80648252-3858-40f9-a636-455cd4e05bf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:15.931475 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80648252-3858-40f9-a636-455cd4e05bf3 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:15.952225 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1268/2537] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:28:15 2022] POST /identity/v3/users => generated 309 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:28:15.957044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff0ef02-357f-456c-8ea2-8bf8d3f20079 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:15.971544 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff0ef02-357f-456c-8ea2-8bf8d3f20079 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:15.977902 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b857d8-c09a-457a-ad1a-33a9de3af1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:15.978574 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b857d8-c09a-457a-ad1a-33a9de3af1d0 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:15.978851 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b857d8-c09a-457a-ad1a-33a9de3af1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:15.979169 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b857d8-c09a-457a-ad1a-33a9de3af1d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:15.979885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b857d8-c09a-457a-ad1a-33a9de3af1d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:15.981841 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b857d8-c09a-457a-ad1a-33a9de3af1d0 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:15.993437 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1270/2538] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:28:15.997731 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80648252-3858-40f9-a636-455cd4e05bf3 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:16.008101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80648252-3858-40f9-a636-455cd4e05bf3 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:16.012788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced0e83-cb06-4865-8e8a-52fa4f8b9059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:16.013772 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced0e83-cb06-4865-8e8a-52fa4f8b9059 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:16.014057 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced0e83-cb06-4865-8e8a-52fa4f8b9059 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:16.014338 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced0e83-cb06-4865-8e8a-52fa4f8b9059 admin admin] PATH_INFO: `/v3/projects/eed48c58bd1240ddba930ec7fbfc1c54/users/be392771d0994c7abcd8f6f6d01817e7/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:16.023098 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:16.030534 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fced0e83-cb06-4865-8e8a-52fa4f8b9059 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eed48c58bd1240ddba930ec7fbfc1c54, user_id=be392771d0994c7abcd8f6f6d01817e7, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:16.033526 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fced0e83-cb06-4865-8e8a-52fa4f8b9059 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:16.046681 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1269/2539] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:28:15 2022] PUT /identity/v3/projects/eed48c58bd1240ddba930ec7fbfc1c54/users/be392771d0994c7abcd8f6f6d01817e7/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:16.050582 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b857d8-c09a-457a-ad1a-33a9de3af1d0 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:16.061034 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b857d8-c09a-457a-ad1a-33a9de3af1d0 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:16.121397 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627c99e-1199-49b5-addc-9f75ed065c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:16.122062 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627c99e-1199-49b5-addc-9f75ed065c19 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:16.122369 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627c99e-1199-49b5-addc-9f75ed065c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:16.122631 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627c99e-1199-49b5-addc-9f75ed065c19 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:16.123390 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3627c99e-1199-49b5-addc-9f75ed065c19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:16.125000 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3627c99e-1199-49b5-addc-9f75ed065c19 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:16.135004 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1271/2540] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:28:16.139409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced0e83-cb06-4865-8e8a-52fa4f8b9059 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:16.151835 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced0e83-cb06-4865-8e8a-52fa4f8b9059 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:16.158760 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91afdff-94a7-4b0f-a9a1-a8fcad2cb86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:16.160176 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91afdff-94a7-4b0f-a9a1-a8fcad2cb86c admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:16.160613 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91afdff-94a7-4b0f-a9a1-a8fcad2cb86c admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:16.161040 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91afdff-94a7-4b0f-a9a1-a8fcad2cb86c admin admin] PATH_INFO: `/v3/projects/eed48c58bd1240ddba930ec7fbfc1c54/users/be392771d0994c7abcd8f6f6d01817e7/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:16.164045 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91afdff-94a7-4b0f-a9a1-a8fcad2cb86c admin admin] RBAC: Authorizing `identity:create_grant(project_id=eed48c58bd1240ddba930ec7fbfc1c54, user_id=be392771d0994c7abcd8f6f6d01817e7, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:16.168437 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91afdff-94a7-4b0f-a9a1-a8fcad2cb86c admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:16.184918 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1270/2541] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:28:16 2022] PUT /identity/v3/projects/eed48c58bd1240ddba930ec7fbfc1c54/users/be392771d0994c7abcd8f6f6d01817e7/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:16.198080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b841f6a-9a5b-45d7-9f65-01300ec3747c None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:16.198377 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b841f6a-9a5b-45d7-9f65-01300ec3747c None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:16.198653 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b841f6a-9a5b-45d7-9f65-01300ec3747c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:16.229349 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-5b841f6a-9a5b-45d7-9f65-01300ec3747c None None] MFA Rules not processed for user `be392771d0994c7abcd8f6f6d01817e7`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:16.230590 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-5b841f6a-9a5b-45d7-9f65-01300ec3747c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:16.304876 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1272/2542] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:28:16 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:16.318873 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91afdff-94a7-4b0f-a9a1-a8fcad2cb86c admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:16.321063 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-a91afdff-94a7-4b0f-a9a1-a8fcad2cb86c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:16.373182 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137418ee-6f3e-4296-9268-ff9a38f6fec5 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:16.373468 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137418ee-6f3e-4296-9268-ff9a38f6fec5 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:16.374237 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137418ee-6f3e-4296-9268-ff9a38f6fec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:16.395720 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-137418ee-6f3e-4296-9268-ff9a38f6fec5 None None] MFA Rules not processed for user `2f8cf6d908314281a3ddc9d9513d39c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:16.396487 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-137418ee-6f3e-4296-9268-ff9a38f6fec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:16.401419 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91afdff-94a7-4b0f-a9a1-a8fcad2cb86c admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:16.405326 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b379322d-9da7-4019-8e73-916d2f71e58a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:16.405993 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379322d-9da7-4019-8e73-916d2f71e58a service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:16.406277 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379322d-9da7-4019-8e73-916d2f71e58a service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:16.406562 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379322d-9da7-4019-8e73-916d2f71e58a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:16.416611 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b379322d-9da7-4019-8e73-916d2f71e58a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:16.418082 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b379322d-9da7-4019-8e73-916d2f71e58a service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:16.430740 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1271/2543] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:28:16 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:28:16.444715 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:16.474359 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1273/2544] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Jul 20 15:28:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:16.490479 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b379322d-9da7-4019-8e73-916d2f71e58a service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:16.492186 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-b379322d-9da7-4019-8e73-916d2f71e58a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:16.590093 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b379322d-9da7-4019-8e73-916d2f71e58a service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:16.596151 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4b7f6-ced3-42e1-a7d3-f02c820ff5bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f4844988474d7a91133d77906611a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:16.596820 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4b7f6-ced3-42e1-a7d3-f02c820ff5bd service swift] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:16.597106 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4b7f6-ced3-42e1-a7d3-f02c820ff5bd service swift] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:16.597381 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4b7f6-ced3-42e1-a7d3-f02c820ff5bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:16.606930 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4b7f6-ced3-42e1-a7d3-f02c820ff5bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:16.608563 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4b7f6-ced3-42e1-a7d3-f02c820ff5bd service swift] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:16.618136 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1272/2545] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jul 20 15:28:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:28:16.627881 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137418ee-6f3e-4296-9268-ff9a38f6fec5 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:16.741550 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137418ee-6f3e-4296-9268-ff9a38f6fec5 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:16.746680 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b54ba-077d-4021-8eb3-c6725ca8fb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:16.748135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b54ba-077d-4021-8eb3-c6725ca8fb17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:16.748135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b54ba-077d-4021-8eb3-c6725ca8fb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:16.750960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b54ba-077d-4021-8eb3-c6725ca8fb17 admin admin] PATH_INFO: `/v3/users/59859d0a15914bc6bc4f063a72d9fa47` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:16.754516 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3b54ba-077d-4021-8eb3-c6725ca8fb17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59859d0a15914bc6bc4f063a72d9fa47)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:16.756592 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3b54ba-077d-4021-8eb3-c6725ca8fb17 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:16.840720 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1c3b54ba-077d-4021-8eb3-c6725ca8fb17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59859d0a15914bc6bc4f063a72d9fa47'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:16.860989 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1c3b54ba-077d-4021-8eb3-c6725ca8fb17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59859d0a15914bc6bc4f063a72d9fa47'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:16.862977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1c3b54ba-077d-4021-8eb3-c6725ca8fb17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59859d0a15914bc6bc4f063a72d9fa47'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:16.864584 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1c3b54ba-077d-4021-8eb3-c6725ca8fb17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59859d0a15914bc6bc4f063a72d9fa47'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:16.876282 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1c3b54ba-077d-4021-8eb3-c6725ca8fb17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59859d0a15914bc6bc4f063a72d9fa47'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:16.913204 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1274/2546] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:28:16 2022] DELETE /identity/v3/users/59859d0a15914bc6bc4f063a72d9fa47 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:16.916066 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6540b-e9c9-4eb3-80a1-719e36cea9c1 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:16.916916 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6540b-e9c9-4eb3-80a1-719e36cea9c1 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:16.917366 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6540b-e9c9-4eb3-80a1-719e36cea9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:16.939403 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-57d6540b-e9c9-4eb3-80a1-719e36cea9c1 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:16.941120 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-57d6540b-e9c9-4eb3-80a1-719e36cea9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:17.016485 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:17.036191 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1273/2547] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:28:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:17.043662 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b54ba-077d-4021-8eb3-c6725ca8fb17 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:17.045412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-1c3b54ba-077d-4021-8eb3-c6725ca8fb17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:17.121988 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b54ba-077d-4021-8eb3-c6725ca8fb17 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:17.130136 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d11a5d-f220-4374-ba38-e800115c3e87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:17.130892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d11a5d-f220-4374-ba38-e800115c3e87 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:17.131277 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d11a5d-f220-4374-ba38-e800115c3e87 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:17.131629 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d11a5d-f220-4374-ba38-e800115c3e87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:17.146526 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d11a5d-f220-4374-ba38-e800115c3e87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:17.148261 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d11a5d-f220-4374-ba38-e800115c3e87 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:17.159878 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1275/2548] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:28:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:28:17.457890 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d6540b-e9c9-4eb3-80a1-719e36cea9c1 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:17.460094 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-57d6540b-e9c9-4eb3-80a1-719e36cea9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:17.473052 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d6540b-e9c9-4eb3-80a1-719e36cea9c1 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:17.497216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b5aef3-7594-4296-a4a1-7aff95104c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:17.501695 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5aef3-7594-4296-a4a1-7aff95104c3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:17.502089 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5aef3-7594-4296-a4a1-7aff95104c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:17.502446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5aef3-7594-4296-a4a1-7aff95104c3d admin admin] PATH_INFO: `/v3/projects/2d4203cf433b4b32b0a465e3e56111b5` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:17.504461 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b5aef3-7594-4296-a4a1-7aff95104c3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d4203cf433b4b32b0a465e3e56111b5)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:17.506927 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b5aef3-7594-4296-a4a1-7aff95104c3d admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:17.567781 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-38b5aef3-7594-4296-a4a1-7aff95104c3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d4203cf433b4b32b0a465e3e56111b5', 'request_id': 'req-38b5aef3-7594-4296-a4a1-7aff95104c3d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:17.573814 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-38b5aef3-7594-4296-a4a1-7aff95104c3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d4203cf433b4b32b0a465e3e56111b5', 'request_id': 'req-38b5aef3-7594-4296-a4a1-7aff95104c3d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:17.580563 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-38b5aef3-7594-4296-a4a1-7aff95104c3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d4203cf433b4b32b0a465e3e56111b5', 'request_id': 'req-38b5aef3-7594-4296-a4a1-7aff95104c3d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:17.594426 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-38b5aef3-7594-4296-a4a1-7aff95104c3d admin admin] The token cache is being invalidate because project 2d4203cf433b4b32b0a465e3e56111b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:28:17.595221 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-38b5aef3-7594-4296-a4a1-7aff95104c3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:17.595862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-38b5aef3-7594-4296-a4a1-7aff95104c3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:17.597453 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1274/2549] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:28:17 2022] DELETE /identity/v3/projects/2d4203cf433b4b32b0a465e3e56111b5 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:17.601795 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d11a5d-f220-4374-ba38-e800115c3e87 service neutron] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:17.612783 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d11a5d-f220-4374-ba38-e800115c3e87 service neutron] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:17.622773 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:17.677930 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5708277-f9f2-4253-a7f3-18327eab9ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:17.678988 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5708277-f9f2-4253-a7f3-18327eab9ba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:17.679362 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5708277-f9f2-4253-a7f3-18327eab9ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:17.679644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5708277-f9f2-4253-a7f3-18327eab9ba8 admin admin] PATH_INFO: `/v3/users/be392771d0994c7abcd8f6f6d01817e7` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:17.680863 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5708277-f9f2-4253-a7f3-18327eab9ba8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be392771d0994c7abcd8f6f6d01817e7)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:17.682800 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5708277-f9f2-4253-a7f3-18327eab9ba8 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:17.736175 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a5708277-f9f2-4253-a7f3-18327eab9ba8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be392771d0994c7abcd8f6f6d01817e7'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:17.753321 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a5708277-f9f2-4253-a7f3-18327eab9ba8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be392771d0994c7abcd8f6f6d01817e7'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:17.754114 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a5708277-f9f2-4253-a7f3-18327eab9ba8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be392771d0994c7abcd8f6f6d01817e7'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:17.754818 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a5708277-f9f2-4253-a7f3-18327eab9ba8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be392771d0994c7abcd8f6f6d01817e7'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:17.763632 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a5708277-f9f2-4253-a7f3-18327eab9ba8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be392771d0994c7abcd8f6f6d01817e7'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:17.780193 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1276/2550] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:28:17 2022] DELETE /identity/v3/users/be392771d0994c7abcd8f6f6d01817e7 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:18.124354 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b5aef3-7594-4296-a4a1-7aff95104c3d admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:18.126357 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-38b5aef3-7594-4296-a4a1-7aff95104c3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:18.146216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b5aef3-7594-4296-a4a1-7aff95104c3d admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:18.233789 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53337dd7-0351-4314-bd0a-8670ad553cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:18.234812 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53337dd7-0351-4314-bd0a-8670ad553cb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:18.235123 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53337dd7-0351-4314-bd0a-8670ad553cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:18.236142 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53337dd7-0351-4314-bd0a-8670ad553cb8 admin admin] PATH_INFO: `/v3/projects/eed48c58bd1240ddba930ec7fbfc1c54` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:18.237391 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53337dd7-0351-4314-bd0a-8670ad553cb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eed48c58bd1240ddba930ec7fbfc1c54)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:18.239425 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53337dd7-0351-4314-bd0a-8670ad553cb8 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:18.286953 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-53337dd7-0351-4314-bd0a-8670ad553cb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eed48c58bd1240ddba930ec7fbfc1c54', 'request_id': 'req-53337dd7-0351-4314-bd0a-8670ad553cb8'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:18.292609 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-53337dd7-0351-4314-bd0a-8670ad553cb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eed48c58bd1240ddba930ec7fbfc1c54', 'request_id': 'req-53337dd7-0351-4314-bd0a-8670ad553cb8'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:18.298826 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-53337dd7-0351-4314-bd0a-8670ad553cb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eed48c58bd1240ddba930ec7fbfc1c54', 'request_id': 'req-53337dd7-0351-4314-bd0a-8670ad553cb8'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:18.313358 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-53337dd7-0351-4314-bd0a-8670ad553cb8 admin admin] The token cache is being invalidate because project eed48c58bd1240ddba930ec7fbfc1c54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:28:18.313655 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-53337dd7-0351-4314-bd0a-8670ad553cb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:18.314394 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-53337dd7-0351-4314-bd0a-8670ad553cb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:18.316097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1275/2551] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:28:18 2022] DELETE /identity/v3/projects/eed48c58bd1240ddba930ec7fbfc1c54 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:18.321069 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea310c-7885-46d8-8518-449cf2e5af84 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:18.321525 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea310c-7885-46d8-8518-449cf2e5af84 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:18.321614 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea310c-7885-46d8-8518-449cf2e5af84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:18.339419 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-9cea310c-7885-46d8-8518-449cf2e5af84 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:18.340646 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-9cea310c-7885-46d8-8518-449cf2e5af84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:18.405182 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1277/2552] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:28:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:18.413768 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcdd89-b085-47df-b816-51b0e7038914 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:18.414215 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcdd89-b085-47df-b816-51b0e7038914 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:18.414497 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcdd89-b085-47df-b816-51b0e7038914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:18.434917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-c2dcdd89-b085-47df-b816-51b0e7038914 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:18.436199 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-c2dcdd89-b085-47df-b816-51b0e7038914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:18.450418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1276/2553] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:28:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:18.454624 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cea310c-7885-46d8-8518-449cf2e5af84 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:18.465423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cea310c-7885-46d8-8518-449cf2e5af84 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:18.470353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63dee22-9553-4432-bde8-e352658fe65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:18.471030 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63dee22-9553-4432-bde8-e352658fe65c admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:18.471341 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63dee22-9553-4432-bde8-e352658fe65c admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:18.471610 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63dee22-9553-4432-bde8-e352658fe65c admin admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:18.472342 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63dee22-9553-4432-bde8-e352658fe65c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:28:18.472650 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63dee22-9553-4432-bde8-e352658fe65c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:18.474286 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63dee22-9553-4432-bde8-e352658fe65c admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:18.488845 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1278/2554] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 15:28:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:28:18.495791 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dcdd89-b085-47df-b816-51b0e7038914 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:18.501187 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:18.508393 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dcdd89-b085-47df-b816-51b0e7038914 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:18.514328 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4693f2-3aab-4c1c-a9e8-55cd592f3def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:18.515001 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4693f2-3aab-4c1c-a9e8-55cd592f3def admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:18.515780 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4693f2-3aab-4c1c-a9e8-55cd592f3def admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:18.516063 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4693f2-3aab-4c1c-a9e8-55cd592f3def admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:18.516770 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4693f2-3aab-4c1c-a9e8-55cd592f3def admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:18.518771 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4693f2-3aab-4c1c-a9e8-55cd592f3def admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:18.551499 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1277/2555] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:28:18 2022] POST /identity/v3/projects => generated 405 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:28:18.558123 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63dee22-9553-4432-bde8-e352658fe65c admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:18.575526 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63dee22-9553-4432-bde8-e352658fe65c admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:18.594581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:18.637598 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38282877-20fc-469f-aed4-3dab760085a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:18.638300 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38282877-20fc-469f-aed4-3dab760085a7 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:18.638588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38282877-20fc-469f-aed4-3dab760085a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:18.638862 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38282877-20fc-469f-aed4-3dab760085a7 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:18.639619 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38282877-20fc-469f-aed4-3dab760085a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:18.641553 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38282877-20fc-469f-aed4-3dab760085a7 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:18.663563 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1279/2556] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:28:18 2022] POST /identity/v3/users => generated 303 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:28:18.667734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4693f2-3aab-4c1c-a9e8-55cd592f3def admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:18.678744 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4693f2-3aab-4c1c-a9e8-55cd592f3def admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:18.683827 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5d15f-dabf-4333-a105-0aea7d9d32f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:18.684564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5d15f-dabf-4333-a105-0aea7d9d32f1 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:18.684856 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5d15f-dabf-4333-a105-0aea7d9d32f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:18.685269 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5d15f-dabf-4333-a105-0aea7d9d32f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:18.686005 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba5d15f-dabf-4333-a105-0aea7d9d32f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:18.687673 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba5d15f-dabf-4333-a105-0aea7d9d32f1 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:18.699246 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1278/2557] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:28:18.703610 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38282877-20fc-469f-aed4-3dab760085a7 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:18.713910 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38282877-20fc-469f-aed4-3dab760085a7 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:18.719192 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2342e4f-653a-4d2c-bba1-494133fcb92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:18.720774 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2342e4f-653a-4d2c-bba1-494133fcb92d admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:18.721274 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2342e4f-653a-4d2c-bba1-494133fcb92d admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:18.721688 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2342e4f-653a-4d2c-bba1-494133fcb92d admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:18.722451 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2342e4f-653a-4d2c-bba1-494133fcb92d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:18.724318 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2342e4f-653a-4d2c-bba1-494133fcb92d admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:18.739686 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1280/2558] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:28:18.741186 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5d15f-dabf-4333-a105-0aea7d9d32f1 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:18.755807 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5d15f-dabf-4333-a105-0aea7d9d32f1 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:18.760807 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c502215-6992-4333-9180-80a53029c5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:18.761718 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c502215-6992-4333-9180-80a53029c5e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:18.762025 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c502215-6992-4333-9180-80a53029c5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:18.762301 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c502215-6992-4333-9180-80a53029c5e7 admin admin] PATH_INFO: `/v3/projects/cad2e758cfd14e348e3a6d6819c87b50/users/f7ba51a41fc540f1b38b46aedea022da/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:18.779574 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c502215-6992-4333-9180-80a53029c5e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cad2e758cfd14e348e3a6d6819c87b50, user_id=f7ba51a41fc540f1b38b46aedea022da, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:18.782413 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c502215-6992-4333-9180-80a53029c5e7 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:18.810591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1279/2559] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:28:18 2022] PUT /identity/v3/projects/cad2e758cfd14e348e3a6d6819c87b50/users/f7ba51a41fc540f1b38b46aedea022da/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:18.815874 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64adb0bb-cb99-4907-957e-5586b2a24a52 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:18.816169 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64adb0bb-cb99-4907-957e-5586b2a24a52 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:18.816491 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64adb0bb-cb99-4907-957e-5586b2a24a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:18.834790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-64adb0bb-cb99-4907-957e-5586b2a24a52 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:18.836041 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-64adb0bb-cb99-4907-957e-5586b2a24a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:18.917655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1281/2560] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:28:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:18.927517 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c502215-6992-4333-9180-80a53029c5e7 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:18.929469 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-2c502215-6992-4333-9180-80a53029c5e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:18.994799 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c502215-6992-4333-9180-80a53029c5e7 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:19.000029 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36849a2-5982-4451-9fe8-b4a944e71af7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:19.000707 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36849a2-5982-4451-9fe8-b4a944e71af7 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:19.000994 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36849a2-5982-4451-9fe8-b4a944e71af7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:19.001436 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36849a2-5982-4451-9fe8-b4a944e71af7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:19.012848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36849a2-5982-4451-9fe8-b4a944e71af7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:19.014486 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36849a2-5982-4451-9fe8-b4a944e71af7 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:19.025694 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1280/2561] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:28:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:28:19.842475 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c754afc-97f7-49d4-8535-c4e02c436c16 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:19.843129 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c754afc-97f7-49d4-8535-c4e02c436c16 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:19.843129 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c754afc-97f7-49d4-8535-c4e02c436c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:19.871767 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-8c754afc-97f7-49d4-8535-c4e02c436c16 None None] MFA Rules not processed for user `f7ba51a41fc540f1b38b46aedea022da`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:19.873034 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-8c754afc-97f7-49d4-8535-c4e02c436c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:19.945420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1282/2562] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:28:19 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:19.949824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36849a2-5982-4451-9fe8-b4a944e71af7 service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:19.962064 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36849a2-5982-4451-9fe8-b4a944e71af7 service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:19.966706 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea42328-947d-4193-9e3e-89164cd1136e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:19.967436 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea42328-947d-4193-9e3e-89164cd1136e admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:19.967723 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea42328-947d-4193-9e3e-89164cd1136e admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:19.968001 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea42328-947d-4193-9e3e-89164cd1136e admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:19.968728 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea42328-947d-4193-9e3e-89164cd1136e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:19.970629 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea42328-947d-4193-9e3e-89164cd1136e admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:20.001266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1281/2563] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:28:19 2022] POST /identity/v3/projects => generated 405 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:28:20.005530 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c754afc-97f7-49d4-8535-c4e02c436c16 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:20.017606 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c754afc-97f7-49d4-8535-c4e02c436c16 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:20.058277 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:20.099239 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac3a4f-18ab-41a1-8a9e-6e72feb1b489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:20.099917 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac3a4f-18ab-41a1-8a9e-6e72feb1b489 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:20.100201 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac3a4f-18ab-41a1-8a9e-6e72feb1b489 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:20.100478 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac3a4f-18ab-41a1-8a9e-6e72feb1b489 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:20.101216 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dac3a4f-18ab-41a1-8a9e-6e72feb1b489 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:20.103168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dac3a4f-18ab-41a1-8a9e-6e72feb1b489 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:20.142455 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1283/2564] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:28:20 2022] POST /identity/v3/users => generated 309 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:28:20.144144 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea42328-947d-4193-9e3e-89164cd1136e admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:20.154856 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea42328-947d-4193-9e3e-89164cd1136e admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:20.159944 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e48e44-101b-4746-93e5-12162bf0040c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:20.160976 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e48e44-101b-4746-93e5-12162bf0040c admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:20.161458 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e48e44-101b-4746-93e5-12162bf0040c admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:20.161787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e48e44-101b-4746-93e5-12162bf0040c admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:20.162523 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e48e44-101b-4746-93e5-12162bf0040c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:20.164166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e48e44-101b-4746-93e5-12162bf0040c admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:20.175306 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1282/2565] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:28:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:28:20.180221 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac3a4f-18ab-41a1-8a9e-6e72feb1b489 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:20.191514 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac3a4f-18ab-41a1-8a9e-6e72feb1b489 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:20.196888 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0879a-d001-47e2-ae49-76fb88595cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:20.197858 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0879a-d001-47e2-ae49-76fb88595cf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:20.198146 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0879a-d001-47e2-ae49-76fb88595cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:20.198421 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0879a-d001-47e2-ae49-76fb88595cf8 admin admin] PATH_INFO: `/v3/projects/446fbdfdc88d42c8b83077d37eb45833/users/19835afc745849248d9282c7a7961305/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:20.217944 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0879a-d001-47e2-ae49-76fb88595cf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=446fbdfdc88d42c8b83077d37eb45833, user_id=19835afc745849248d9282c7a7961305, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:20.220800 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0879a-d001-47e2-ae49-76fb88595cf8 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:20.236163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1284/2566] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:28:20 2022] PUT /identity/v3/projects/446fbdfdc88d42c8b83077d37eb45833/users/19835afc745849248d9282c7a7961305/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:20.240326 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e48e44-101b-4746-93e5-12162bf0040c admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:20.253427 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e48e44-101b-4746-93e5-12162bf0040c admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:20.376586 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1998b929-f5ea-4eb9-96c4-8ebcd820c0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:20.377293 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998b929-f5ea-4eb9-96c4-8ebcd820c0f2 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:20.377572 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998b929-f5ea-4eb9-96c4-8ebcd820c0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:20.377904 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998b929-f5ea-4eb9-96c4-8ebcd820c0f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:20.378628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1998b929-f5ea-4eb9-96c4-8ebcd820c0f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:20.380284 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1998b929-f5ea-4eb9-96c4-8ebcd820c0f2 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:20.404239 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1283/2567] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:28:20 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:28:20.409072 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0879a-d001-47e2-ae49-76fb88595cf8 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:20.431730 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0879a-d001-47e2-ae49-76fb88595cf8 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:20.440161 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25c109-8d29-4b27-b3d8-e873cf5d666b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:20.442842 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25c109-8d29-4b27-b3d8-e873cf5d666b admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:20.443168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25c109-8d29-4b27-b3d8-e873cf5d666b admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:20.443446 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25c109-8d29-4b27-b3d8-e873cf5d666b admin admin] PATH_INFO: `/v3/projects/446fbdfdc88d42c8b83077d37eb45833/users/19835afc745849248d9282c7a7961305/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:20.450512 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef25c109-8d29-4b27-b3d8-e873cf5d666b admin admin] RBAC: Authorizing `identity:create_grant(project_id=446fbdfdc88d42c8b83077d37eb45833, user_id=19835afc745849248d9282c7a7961305, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:20.464096 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef25c109-8d29-4b27-b3d8-e873cf5d666b admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:20.477626 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1285/2568] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:28:20 2022] PUT /identity/v3/projects/446fbdfdc88d42c8b83077d37eb45833/users/19835afc745849248d9282c7a7961305/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:21.292758 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57d4bb-4347-4bab-b5aa-deed193dccf8 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:21.293859 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57d4bb-4347-4bab-b5aa-deed193dccf8 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:21.294678 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57d4bb-4347-4bab-b5aa-deed193dccf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:21.316385 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:21.326960 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-5e57d4bb-4347-4bab-b5aa-deed193dccf8 None None] MFA Rules not processed for user `19835afc745849248d9282c7a7961305`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:21.329865 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-5e57d4bb-4347-4bab-b5aa-deed193dccf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:21.415249 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1284/2569] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:28:21 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:22.611637 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25c109-8d29-4b27-b3d8-e873cf5d666b admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:22.684391 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25c109-8d29-4b27-b3d8-e873cf5d666b admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:22.689456 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ac0fd-a8da-42f6-82d2-ddbac864f42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:22.690351 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ac0fd-a8da-42f6-82d2-ddbac864f42d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:22.690633 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ac0fd-a8da-42f6-82d2-ddbac864f42d admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:22.690912 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ac0fd-a8da-42f6-82d2-ddbac864f42d admin admin] PATH_INFO: `/v3/users/f7ba51a41fc540f1b38b46aedea022da` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:22.692190 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07ac0fd-a8da-42f6-82d2-ddbac864f42d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7ba51a41fc540f1b38b46aedea022da)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:22.694254 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07ac0fd-a8da-42f6-82d2-ddbac864f42d admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:22.772052 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-c07ac0fd-a8da-42f6-82d2-ddbac864f42d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7ba51a41fc540f1b38b46aedea022da'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:22.795261 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-c07ac0fd-a8da-42f6-82d2-ddbac864f42d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7ba51a41fc540f1b38b46aedea022da'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:22.795997 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-c07ac0fd-a8da-42f6-82d2-ddbac864f42d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7ba51a41fc540f1b38b46aedea022da'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:22.796613 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-c07ac0fd-a8da-42f6-82d2-ddbac864f42d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7ba51a41fc540f1b38b46aedea022da'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:22.809011 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-c07ac0fd-a8da-42f6-82d2-ddbac864f42d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7ba51a41fc540f1b38b46aedea022da'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:22.823929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:22.829126 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1286/2570] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:28:22 2022] DELETE /identity/v3/users/f7ba51a41fc540f1b38b46aedea022da => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:28:23.020215 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57d4bb-4347-4bab-b5aa-deed193dccf8 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:23.043544 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57d4bb-4347-4bab-b5aa-deed193dccf8 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:23.139316 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d23018-f440-402a-af49-b1f4f1e53598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:23.141579 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d23018-f440-402a-af49-b1f4f1e53598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:23.141992 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d23018-f440-402a-af49-b1f4f1e53598 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:23.142339 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d23018-f440-402a-af49-b1f4f1e53598 admin admin] PATH_INFO: `/v3/projects/cad2e758cfd14e348e3a6d6819c87b50` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:23.145193 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d23018-f440-402a-af49-b1f4f1e53598 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cad2e758cfd14e348e3a6d6819c87b50)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:23.147324 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d23018-f440-402a-af49-b1f4f1e53598 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:23.203543 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-97d23018-f440-402a-af49-b1f4f1e53598 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cad2e758cfd14e348e3a6d6819c87b50', 'request_id': 'req-97d23018-f440-402a-af49-b1f4f1e53598'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:23.209559 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-97d23018-f440-402a-af49-b1f4f1e53598 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cad2e758cfd14e348e3a6d6819c87b50', 'request_id': 'req-97d23018-f440-402a-af49-b1f4f1e53598'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:23.216930 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-97d23018-f440-402a-af49-b1f4f1e53598 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cad2e758cfd14e348e3a6d6819c87b50', 'request_id': 'req-97d23018-f440-402a-af49-b1f4f1e53598'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:23.236902 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-97d23018-f440-402a-af49-b1f4f1e53598 admin admin] The token cache is being invalidate because project cad2e758cfd14e348e3a6d6819c87b50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:28:23.237410 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-97d23018-f440-402a-af49-b1f4f1e53598 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:23.238091 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-97d23018-f440-402a-af49-b1f4f1e53598 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:23.240324 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1285/2571] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:28:23 2022] DELETE /identity/v3/projects/cad2e758cfd14e348e3a6d6819c87b50 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:23.244022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ac0fd-a8da-42f6-82d2-ddbac864f42d admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:23.245464 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-c07ac0fd-a8da-42f6-82d2-ddbac864f42d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:23.260356 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ac0fd-a8da-42f6-82d2-ddbac864f42d admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:23.327811 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d45d35-8ac2-4a5f-906b-1d980e3e1857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:23.329029 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d45d35-8ac2-4a5f-906b-1d980e3e1857 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:23.329557 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d45d35-8ac2-4a5f-906b-1d980e3e1857 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:23.330120 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d45d35-8ac2-4a5f-906b-1d980e3e1857 admin admin] PATH_INFO: `/v3/users/19835afc745849248d9282c7a7961305` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:23.331592 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d45d35-8ac2-4a5f-906b-1d980e3e1857 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19835afc745849248d9282c7a7961305)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:23.333879 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d45d35-8ac2-4a5f-906b-1d980e3e1857 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:23.388090 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-35d45d35-8ac2-4a5f-906b-1d980e3e1857 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19835afc745849248d9282c7a7961305'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:23.403460 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-35d45d35-8ac2-4a5f-906b-1d980e3e1857 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19835afc745849248d9282c7a7961305'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:23.404226 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-35d45d35-8ac2-4a5f-906b-1d980e3e1857 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19835afc745849248d9282c7a7961305'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:23.404830 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-35d45d35-8ac2-4a5f-906b-1d980e3e1857 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19835afc745849248d9282c7a7961305'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:23.412785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-35d45d35-8ac2-4a5f-906b-1d980e3e1857 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19835afc745849248d9282c7a7961305'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:23.447957 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1287/2572] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:28:23 2022] DELETE /identity/v3/users/19835afc745849248d9282c7a7961305 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:28:23.640078 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d23018-f440-402a-af49-b1f4f1e53598 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:23.641723 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-97d23018-f440-402a-af49-b1f4f1e53598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:23.654692 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d23018-f440-402a-af49-b1f4f1e53598 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:23.694275 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:23.730608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d424a-a684-4359-a2d2-fde117758cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:23.731834 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d424a-a684-4359-a2d2-fde117758cad admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:23.732115 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d424a-a684-4359-a2d2-fde117758cad admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:23.732392 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d424a-a684-4359-a2d2-fde117758cad admin admin] PATH_INFO: `/v3/projects/446fbdfdc88d42c8b83077d37eb45833` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:23.733924 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4d424a-a684-4359-a2d2-fde117758cad admin admin] RBAC: Authorizing `identity:delete_project(project_id=446fbdfdc88d42c8b83077d37eb45833)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:23.735871 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4d424a-a684-4359-a2d2-fde117758cad admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:23.786798 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-5f4d424a-a684-4359-a2d2-fde117758cad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '446fbdfdc88d42c8b83077d37eb45833', 'request_id': 'req-5f4d424a-a684-4359-a2d2-fde117758cad'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:23.792717 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-5f4d424a-a684-4359-a2d2-fde117758cad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '446fbdfdc88d42c8b83077d37eb45833', 'request_id': 'req-5f4d424a-a684-4359-a2d2-fde117758cad'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:23.799174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-5f4d424a-a684-4359-a2d2-fde117758cad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '446fbdfdc88d42c8b83077d37eb45833', 'request_id': 'req-5f4d424a-a684-4359-a2d2-fde117758cad'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:23.813665 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-5f4d424a-a684-4359-a2d2-fde117758cad admin admin] The token cache is being invalidate because project 446fbdfdc88d42c8b83077d37eb45833 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:28:23.814172 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-5f4d424a-a684-4359-a2d2-fde117758cad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:23.814989 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-5f4d424a-a684-4359-a2d2-fde117758cad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:23.820097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1286/2573] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:28:23 2022] DELETE /identity/v3/projects/446fbdfdc88d42c8b83077d37eb45833 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:23.821877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eafe2b-d7d3-4895-8431-988f6ce6f390 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:23.822401 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eafe2b-d7d3-4895-8431-988f6ce6f390 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:23.822523 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eafe2b-d7d3-4895-8431-988f6ce6f390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:23.843221 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-76eafe2b-d7d3-4895-8431-988f6ce6f390 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:23.844595 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-76eafe2b-d7d3-4895-8431-988f6ce6f390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:23.922004 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1288/2574] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:28:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:23.930962 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40485f-363b-4094-84cf-1f5a25d6d1ee None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:23.931313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40485f-363b-4094-84cf-1f5a25d6d1ee None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:23.931630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40485f-363b-4094-84cf-1f5a25d6d1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:23.952221 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-9f40485f-363b-4094-84cf-1f5a25d6d1ee None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:23.953993 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-9f40485f-363b-4094-84cf-1f5a25d6d1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:23.963396 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1287/2575] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:28:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:23.967946 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eafe2b-d7d3-4895-8431-988f6ce6f390 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:23.978393 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eafe2b-d7d3-4895-8431-988f6ce6f390 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:23.984352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14a2d7-2d89-495a-ae98-81f8653e0f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:23.985170 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14a2d7-2d89-495a-ae98-81f8653e0f95 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:23.985465 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14a2d7-2d89-495a-ae98-81f8653e0f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:23.985735 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14a2d7-2d89-495a-ae98-81f8653e0f95 admin admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:23.986499 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14a2d7-2d89-495a-ae98-81f8653e0f95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:28:23.986802 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14a2d7-2d89-495a-ae98-81f8653e0f95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:23.988443 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14a2d7-2d89-495a-ae98-81f8653e0f95 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:24.002302 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1289/2576] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 15:28:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:28:24.007111 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f40485f-363b-4094-84cf-1f5a25d6d1ee None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:24.021114 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f40485f-363b-4094-84cf-1f5a25d6d1ee None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:24.026195 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4f4ac-e72b-4a00-a0ac-5e520801bb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:24.026876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4f4ac-e72b-4a00-a0ac-5e520801bb5b admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:24.027226 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4f4ac-e72b-4a00-a0ac-5e520801bb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:24.027511 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4f4ac-e72b-4a00-a0ac-5e520801bb5b admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:24.028217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4f4ac-e72b-4a00-a0ac-5e520801bb5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:24.030382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4f4ac-e72b-4a00-a0ac-5e520801bb5b admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:24.068737 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1288/2577] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:28:24 2022] POST /identity/v3/projects => generated 381 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:28:24.073060 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14a2d7-2d89-495a-ae98-81f8653e0f95 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:24.086300 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14a2d7-2d89-495a-ae98-81f8653e0f95 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:24.090711 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:24.149381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce353578-ef97-431c-a311-a99484c25c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:24.150063 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce353578-ef97-431c-a311-a99484c25c99 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:24.150354 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce353578-ef97-431c-a311-a99484c25c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:24.150636 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce353578-ef97-431c-a311-a99484c25c99 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:24.151382 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce353578-ef97-431c-a311-a99484c25c99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:24.153308 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce353578-ef97-431c-a311-a99484c25c99 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:24.176089 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1290/2578] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:28:24 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:28:24.180207 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4f4ac-e72b-4a00-a0ac-5e520801bb5b admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:24.191315 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4f4ac-e72b-4a00-a0ac-5e520801bb5b admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:24.196232 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef39124-ce58-4a53-b3b9-36ef9b0a935a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:24.196918 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef39124-ce58-4a53-b3b9-36ef9b0a935a admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:24.197206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef39124-ce58-4a53-b3b9-36ef9b0a935a admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:24.197486 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef39124-ce58-4a53-b3b9-36ef9b0a935a admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:24.198211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef39124-ce58-4a53-b3b9-36ef9b0a935a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:24.199876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef39124-ce58-4a53-b3b9-36ef9b0a935a admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:24.211076 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1289/2579] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:28:24.215198 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce353578-ef97-431c-a311-a99484c25c99 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:24.225646 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce353578-ef97-431c-a311-a99484c25c99 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:24.230343 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1053b7f0-7c2c-420d-9594-61cce86a42fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:24.231361 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053b7f0-7c2c-420d-9594-61cce86a42fc admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:24.231699 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053b7f0-7c2c-420d-9594-61cce86a42fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:24.231981 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053b7f0-7c2c-420d-9594-61cce86a42fc admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:24.232719 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1053b7f0-7c2c-420d-9594-61cce86a42fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:24.234334 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1053b7f0-7c2c-420d-9594-61cce86a42fc admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:24.244576 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1291/2580] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:28:24.249651 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef39124-ce58-4a53-b3b9-36ef9b0a935a admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:24.260837 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef39124-ce58-4a53-b3b9-36ef9b0a935a admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:24.266085 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3889a8-9dd7-48d9-a799-23f4efa32bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:24.267103 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3889a8-9dd7-48d9-a799-23f4efa32bdc admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:24.267397 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3889a8-9dd7-48d9-a799-23f4efa32bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:24.267687 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3889a8-9dd7-48d9-a799-23f4efa32bdc admin admin] PATH_INFO: `/v3/projects/ab763495787c41b68a97ce47d6b26b37/users/0bba3f71e0994b5383a6cdebca4db770/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:24.286787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3889a8-9dd7-48d9-a799-23f4efa32bdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab763495787c41b68a97ce47d6b26b37, user_id=0bba3f71e0994b5383a6cdebca4db770, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:24.289645 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3889a8-9dd7-48d9-a799-23f4efa32bdc admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:24.315452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1290/2581] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:28:24 2022] PUT /identity/v3/projects/ab763495787c41b68a97ce47d6b26b37/users/0bba3f71e0994b5383a6cdebca4db770/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:24.325202 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2baa05f-5788-4550-ba9d-e04906a489b7 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:24.325533 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2baa05f-5788-4550-ba9d-e04906a489b7 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:24.325823 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2baa05f-5788-4550-ba9d-e04906a489b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:24.382596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-e2baa05f-5788-4550-ba9d-e04906a489b7 None None] MFA Rules not processed for user `0bba3f71e0994b5383a6cdebca4db770`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:24.384869 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-e2baa05f-5788-4550-ba9d-e04906a489b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:24.465197 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1292/2582] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:28:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:24.469792 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3889a8-9dd7-48d9-a799-23f4efa32bdc admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:24.532649 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:24.544116 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3889a8-9dd7-48d9-a799-23f4efa32bdc admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:24.552658 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279f68a-0980-4f17-94bf-cbbeab604080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:24.553342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279f68a-0980-4f17-94bf-cbbeab604080 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:24.553637 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279f68a-0980-4f17-94bf-cbbeab604080 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:24.553918 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279f68a-0980-4f17-94bf-cbbeab604080 admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:24.554642 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279f68a-0980-4f17-94bf-cbbeab604080 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:24.556601 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279f68a-0980-4f17-94bf-cbbeab604080 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:24.582741 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1291/2583] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:28:24 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:28:24.586976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2baa05f-5788-4550-ba9d-e04906a489b7 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:24.598316 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2baa05f-5788-4550-ba9d-e04906a489b7 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:24.656200 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f1447-2858-4319-984e-ae456d8b8757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:24.656889 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f1447-2858-4319-984e-ae456d8b8757 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:24.657170 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f1447-2858-4319-984e-ae456d8b8757 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:24.657451 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f1447-2858-4319-984e-ae456d8b8757 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:24.658152 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f1447-2858-4319-984e-ae456d8b8757 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:24.660249 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f1447-2858-4319-984e-ae456d8b8757 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:24.682132 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1293/2584] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:28:24 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:28:24.686281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279f68a-0980-4f17-94bf-cbbeab604080 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:24.697630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279f68a-0980-4f17-94bf-cbbeab604080 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:24.703127 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c1f57e-9848-47b3-ac5a-4459f8dc4729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:24.703812 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1f57e-9848-47b3-ac5a-4459f8dc4729 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:24.704097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1f57e-9848-47b3-ac5a-4459f8dc4729 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:24.704378 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1f57e-9848-47b3-ac5a-4459f8dc4729 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:24.705148 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c1f57e-9848-47b3-ac5a-4459f8dc4729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:24.706949 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c1f57e-9848-47b3-ac5a-4459f8dc4729 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:24.718848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1292/2585] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:28:24.723351 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f1447-2858-4319-984e-ae456d8b8757 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:24.730409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:24.735162 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f1447-2858-4319-984e-ae456d8b8757 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:24.742524 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4776bda-91f1-4b16-aa17-6b8a96f069d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:24.743463 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4776bda-91f1-4b16-aa17-6b8a96f069d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:24.743755 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4776bda-91f1-4b16-aa17-6b8a96f069d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:24.744174 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4776bda-91f1-4b16-aa17-6b8a96f069d8 admin admin] PATH_INFO: `/v3/projects/a778f7c454c242829f185369577d8424/users/7bd2a1e4e50343b28c2f9646abb7e696/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:24.760738 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4776bda-91f1-4b16-aa17-6b8a96f069d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a778f7c454c242829f185369577d8424, user_id=7bd2a1e4e50343b28c2f9646abb7e696, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:24.763515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4776bda-91f1-4b16-aa17-6b8a96f069d8 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:24.780987 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1294/2586] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:28:24 2022] PUT /identity/v3/projects/a778f7c454c242829f185369577d8424/users/7bd2a1e4e50343b28c2f9646abb7e696/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:24.785286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c1f57e-9848-47b3-ac5a-4459f8dc4729 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:24.795873 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c1f57e-9848-47b3-ac5a-4459f8dc4729 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:24.857298 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea32ab-ec9f-45cc-a156-51b89caa80c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:24.857902 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea32ab-ec9f-45cc-a156-51b89caa80c6 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:24.858188 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea32ab-ec9f-45cc-a156-51b89caa80c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:24.858464 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea32ab-ec9f-45cc-a156-51b89caa80c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:24.859259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea32ab-ec9f-45cc-a156-51b89caa80c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:24.860918 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea32ab-ec9f-45cc-a156-51b89caa80c6 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:24.871672 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1293/2587] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:28:24.875945 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4776bda-91f1-4b16-aa17-6b8a96f069d8 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:24.887693 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4776bda-91f1-4b16-aa17-6b8a96f069d8 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:24.892740 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863bf004-9cca-4cee-8b15-5c8834b817cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:24.893730 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bf004-9cca-4cee-8b15-5c8834b817cd admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:24.894030 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bf004-9cca-4cee-8b15-5c8834b817cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:24.894294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bf004-9cca-4cee-8b15-5c8834b817cd admin admin] PATH_INFO: `/v3/projects/a778f7c454c242829f185369577d8424/users/7bd2a1e4e50343b28c2f9646abb7e696/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:24.896655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863bf004-9cca-4cee-8b15-5c8834b817cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a778f7c454c242829f185369577d8424, user_id=7bd2a1e4e50343b28c2f9646abb7e696, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:24.899615 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863bf004-9cca-4cee-8b15-5c8834b817cd admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:24.919786 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1295/2588] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:28:24 2022] PUT /identity/v3/projects/a778f7c454c242829f185369577d8424/users/7bd2a1e4e50343b28c2f9646abb7e696/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:24.922320 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639a945-b1c1-4f10-9f3f-afa8fdca5394 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:24.922683 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639a945-b1c1-4f10-9f3f-afa8fdca5394 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:24.923028 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639a945-b1c1-4f10-9f3f-afa8fdca5394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:24.952515 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-7639a945-b1c1-4f10-9f3f-afa8fdca5394 None None] MFA Rules not processed for user `7bd2a1e4e50343b28c2f9646abb7e696`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:24.953822 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-7639a945-b1c1-4f10-9f3f-afa8fdca5394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:25.036518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1294/2589] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:28:24 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:25.044207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863bf004-9cca-4cee-8b15-5c8834b817cd admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:25.046131 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-863bf004-9cca-4cee-8b15-5c8834b817cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:25.113710 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863bf004-9cca-4cee-8b15-5c8834b817cd admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:25.117698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc7410-7c02-48d1-af69-3347ba866a98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:25.118369 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc7410-7c02-48d1-af69-3347ba866a98 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:25.118687 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc7410-7c02-48d1-af69-3347ba866a98 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:25.118937 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc7410-7c02-48d1-af69-3347ba866a98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:25.129268 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbc7410-7c02-48d1-af69-3347ba866a98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:25.130729 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbc7410-7c02-48d1-af69-3347ba866a98 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:25.140973 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1296/2590] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:28:25 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:28:25.226743 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7639a945-b1c1-4f10-9f3f-afa8fdca5394 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:25.238420 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7639a945-b1c1-4f10-9f3f-afa8fdca5394 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:25.242366 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c36912e-9dc0-4c8f-ad55-ca2051e4affe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:25.243279 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c36912e-9dc0-4c8f-ad55-ca2051e4affe service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:25.243746 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c36912e-9dc0-4c8f-ad55-ca2051e4affe service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:25.244188 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c36912e-9dc0-4c8f-ad55-ca2051e4affe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:25.312566 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:25.327879 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c36912e-9dc0-4c8f-ad55-ca2051e4affe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:25.329695 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c36912e-9dc0-4c8f-ad55-ca2051e4affe service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:25.340120 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1295/2591] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:28:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:28:35.136406 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=39,keystone:INSERT=2 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:35.335800 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=6 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:43.621020 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc7410-7c02-48d1-af69-3347ba866a98 service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:43.686138 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc7410-7c02-48d1-af69-3347ba866a98 service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:43.692194 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6995c-c262-4550-8a4d-138f0166335e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:43.693204 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6995c-c262-4550-8a4d-138f0166335e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:43.693494 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6995c-c262-4550-8a4d-138f0166335e admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:43.693778 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6995c-c262-4550-8a4d-138f0166335e admin admin] PATH_INFO: `/v3/users/0bba3f71e0994b5383a6cdebca4db770` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:43.695022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc6995c-c262-4550-8a4d-138f0166335e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bba3f71e0994b5383a6cdebca4db770)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:43.696988 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc6995c-c262-4550-8a4d-138f0166335e admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:43.745742 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-6dc6995c-c262-4550-8a4d-138f0166335e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bba3f71e0994b5383a6cdebca4db770'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:43.757410 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-6dc6995c-c262-4550-8a4d-138f0166335e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bba3f71e0994b5383a6cdebca4db770'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:43.758186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-6dc6995c-c262-4550-8a4d-138f0166335e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bba3f71e0994b5383a6cdebca4db770'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:43.758783 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-6dc6995c-c262-4550-8a4d-138f0166335e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bba3f71e0994b5383a6cdebca4db770'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:43.767610 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-6dc6995c-c262-4550-8a4d-138f0166335e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bba3f71e0994b5383a6cdebca4db770'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:43.782432 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1297/2592] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:28:43 2022] DELETE /identity/v3/users/0bba3f71e0994b5383a6cdebca4db770 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:43.787316 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4fdd99-1b66-42c3-a439-a2b86e933b0a None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:43.787802 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4fdd99-1b66-42c3-a439-a2b86e933b0a None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:43.788077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4fdd99-1b66-42c3-a439-a2b86e933b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:43.807291 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-bb4fdd99-1b66-42c3-a439-a2b86e933b0a None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:43.808527 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-bb4fdd99-1b66-42c3-a439-a2b86e933b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:43.867880 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1296/2593] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:28:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:43.878515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6995c-c262-4550-8a4d-138f0166335e admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:43.881027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-6dc6995c-c262-4550-8a4d-138f0166335e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:43.942469 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6995c-c262-4550-8a4d-138f0166335e admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:43.946508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d707a-5e04-474e-bffa-47aeea3776bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:43.947193 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d707a-5e04-474e-bffa-47aeea3776bf service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:43.947499 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d707a-5e04-474e-bffa-47aeea3776bf service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:43.947754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d707a-5e04-474e-bffa-47aeea3776bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:43.956887 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d707a-5e04-474e-bffa-47aeea3776bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:43.958361 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d707a-5e04-474e-bffa-47aeea3776bf service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:43.968350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1298/2594] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:28:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:28:44.269536 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4fdd99-1b66-42c3-a439-a2b86e933b0a None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:44.270949 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-bb4fdd99-1b66-42c3-a439-a2b86e933b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:44.283567 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4fdd99-1b66-42c3-a439-a2b86e933b0a None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:44.288805 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c7673-d0a5-4b90-979a-f3f8139bb6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:44.289666 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c7673-d0a5-4b90-979a-f3f8139bb6dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:44.289953 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c7673-d0a5-4b90-979a-f3f8139bb6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:44.290237 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c7673-d0a5-4b90-979a-f3f8139bb6dc admin admin] PATH_INFO: `/v3/projects/ab763495787c41b68a97ce47d6b26b37` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:44.291520 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c7673-d0a5-4b90-979a-f3f8139bb6dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab763495787c41b68a97ce47d6b26b37)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:44.293438 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c7673-d0a5-4b90-979a-f3f8139bb6dc admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:44.354531 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-d96c7673-d0a5-4b90-979a-f3f8139bb6dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab763495787c41b68a97ce47d6b26b37', 'request_id': 'req-d96c7673-d0a5-4b90-979a-f3f8139bb6dc'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:44.360976 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-d96c7673-d0a5-4b90-979a-f3f8139bb6dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab763495787c41b68a97ce47d6b26b37', 'request_id': 'req-d96c7673-d0a5-4b90-979a-f3f8139bb6dc'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:44.367609 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-d96c7673-d0a5-4b90-979a-f3f8139bb6dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab763495787c41b68a97ce47d6b26b37', 'request_id': 'req-d96c7673-d0a5-4b90-979a-f3f8139bb6dc'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:44.381195 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-d96c7673-d0a5-4b90-979a-f3f8139bb6dc admin admin] The token cache is being invalidate because project ab763495787c41b68a97ce47d6b26b37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:28:44.381481 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-d96c7673-d0a5-4b90-979a-f3f8139bb6dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:44.382114 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-d96c7673-d0a5-4b90-979a-f3f8139bb6dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:44.383705 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1297/2595] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:28:44 2022] DELETE /identity/v3/projects/ab763495787c41b68a97ce47d6b26b37 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:44.387831 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d707a-5e04-474e-bffa-47aeea3776bf service neutron] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:44.398101 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d707a-5e04-474e-bffa-47aeea3776bf service neutron] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:44.417780 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:44.465473 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df291035-ad7c-4d1d-95b9-2e498b712439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:44.466255 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df291035-ad7c-4d1d-95b9-2e498b712439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:44.466592 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df291035-ad7c-4d1d-95b9-2e498b712439 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:44.466877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df291035-ad7c-4d1d-95b9-2e498b712439 admin admin] PATH_INFO: `/v3/users/7bd2a1e4e50343b28c2f9646abb7e696` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:44.468170 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df291035-ad7c-4d1d-95b9-2e498b712439 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bd2a1e4e50343b28c2f9646abb7e696)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:44.470182 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df291035-ad7c-4d1d-95b9-2e498b712439 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:44.527576 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-df291035-ad7c-4d1d-95b9-2e498b712439 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bd2a1e4e50343b28c2f9646abb7e696'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:44.543877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-df291035-ad7c-4d1d-95b9-2e498b712439 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bd2a1e4e50343b28c2f9646abb7e696'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:44.544632 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-df291035-ad7c-4d1d-95b9-2e498b712439 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bd2a1e4e50343b28c2f9646abb7e696'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:44.545397 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-df291035-ad7c-4d1d-95b9-2e498b712439 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bd2a1e4e50343b28c2f9646abb7e696'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:44.553405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-df291035-ad7c-4d1d-95b9-2e498b712439 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bd2a1e4e50343b28c2f9646abb7e696'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:44.569178 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1299/2596] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:28:44 2022] DELETE /identity/v3/users/7bd2a1e4e50343b28c2f9646abb7e696 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:44.851581 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c7673-d0a5-4b90-979a-f3f8139bb6dc admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:44.854129 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-d96c7673-d0a5-4b90-979a-f3f8139bb6dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:44.876112 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c7673-d0a5-4b90-979a-f3f8139bb6dc admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:44.946113 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb0d45-493b-46ae-8720-53cf5ce66686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:44.946890 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb0d45-493b-46ae-8720-53cf5ce66686 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:44.947235 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb0d45-493b-46ae-8720-53cf5ce66686 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:44.947696 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb0d45-493b-46ae-8720-53cf5ce66686 admin admin] PATH_INFO: `/v3/projects/a778f7c454c242829f185369577d8424` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:44.950426 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cb0d45-493b-46ae-8720-53cf5ce66686 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a778f7c454c242829f185369577d8424)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:44.952397 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cb0d45-493b-46ae-8720-53cf5ce66686 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:44.989164 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:45.010588 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-94cb0d45-493b-46ae-8720-53cf5ce66686 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a778f7c454c242829f185369577d8424', 'request_id': 'req-94cb0d45-493b-46ae-8720-53cf5ce66686'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:45.016983 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-94cb0d45-493b-46ae-8720-53cf5ce66686 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a778f7c454c242829f185369577d8424', 'request_id': 'req-94cb0d45-493b-46ae-8720-53cf5ce66686'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:45.023700 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-94cb0d45-493b-46ae-8720-53cf5ce66686 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a778f7c454c242829f185369577d8424', 'request_id': 'req-94cb0d45-493b-46ae-8720-53cf5ce66686'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:45.036693 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-94cb0d45-493b-46ae-8720-53cf5ce66686 admin admin] The token cache is being invalidate because project a778f7c454c242829f185369577d8424 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:28:45.036990 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-94cb0d45-493b-46ae-8720-53cf5ce66686 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:45.037612 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-94cb0d45-493b-46ae-8720-53cf5ce66686 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:45.039473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1298/2597] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:28:44 2022] DELETE /identity/v3/projects/a778f7c454c242829f185369577d8424 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:45.044444 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d87a8e-da1a-4c20-ade2-e96d1883f182 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:45.044906 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d87a8e-da1a-4c20-ade2-e96d1883f182 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:45.045026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d87a8e-da1a-4c20-ade2-e96d1883f182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:45.062670 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-22d87a8e-da1a-4c20-ade2-e96d1883f182 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:45.063929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-22d87a8e-da1a-4c20-ade2-e96d1883f182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:45.133374 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1300/2598] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:28:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:45.141486 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8facc82-85a0-43da-924a-01186f66fdb6 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:45.141781 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8facc82-85a0-43da-924a-01186f66fdb6 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:45.142156 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8facc82-85a0-43da-924a-01186f66fdb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:45.161548 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-b8facc82-85a0-43da-924a-01186f66fdb6 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:45.163386 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-b8facc82-85a0-43da-924a-01186f66fdb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:45.179084 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1299/2599] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:28:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:45.180460 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d87a8e-da1a-4c20-ade2-e96d1883f182 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:45.196030 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d87a8e-da1a-4c20-ade2-e96d1883f182 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:45.202763 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919307d3-f0fe-48d5-a3c4-940c7a834d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:45.203469 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919307d3-f0fe-48d5-a3c4-940c7a834d2e admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:45.203753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919307d3-f0fe-48d5-a3c4-940c7a834d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:45.204063 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919307d3-f0fe-48d5-a3c4-940c7a834d2e admin admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:45.204751 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919307d3-f0fe-48d5-a3c4-940c7a834d2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:28:45.205064 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919307d3-f0fe-48d5-a3c4-940c7a834d2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:45.206687 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919307d3-f0fe-48d5-a3c4-940c7a834d2e admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:45.219348 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1301/2600] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 15:28:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:28:45.223863 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8facc82-85a0-43da-924a-01186f66fdb6 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:45.234144 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8facc82-85a0-43da-924a-01186f66fdb6 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:45.239097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b817dc-409e-4541-95ac-36e1ba765cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:45.239935 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b817dc-409e-4541-95ac-36e1ba765cda admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:45.240281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b817dc-409e-4541-95ac-36e1ba765cda admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:45.240651 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b817dc-409e-4541-95ac-36e1ba765cda admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:45.241462 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b817dc-409e-4541-95ac-36e1ba765cda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:45.243703 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b817dc-409e-4541-95ac-36e1ba765cda admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:45.282304 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1300/2601] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:28:45 2022] POST /identity/v3/projects => generated 387 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:28:45.286524 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919307d3-f0fe-48d5-a3c4-940c7a834d2e admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:45.297232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919307d3-f0fe-48d5-a3c4-940c7a834d2e admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:45.326633 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:45.366274 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ebe35-d15e-46ac-afda-2dd4862acd1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:45.366962 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ebe35-d15e-46ac-afda-2dd4862acd1a admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:45.367273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ebe35-d15e-46ac-afda-2dd4862acd1a admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:45.367545 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ebe35-d15e-46ac-afda-2dd4862acd1a admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:45.368248 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ebe35-d15e-46ac-afda-2dd4862acd1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:45.370317 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ebe35-d15e-46ac-afda-2dd4862acd1a admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:45.397343 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1302/2602] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:28:45 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:28:45.398546 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b817dc-409e-4541-95ac-36e1ba765cda admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:45.409649 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b817dc-409e-4541-95ac-36e1ba765cda admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:45.414372 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64285ad3-8d5e-4b92-9088-48c8d527b591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:45.415089 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64285ad3-8d5e-4b92-9088-48c8d527b591 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:45.415355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64285ad3-8d5e-4b92-9088-48c8d527b591 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:45.415634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64285ad3-8d5e-4b92-9088-48c8d527b591 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:45.416365 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64285ad3-8d5e-4b92-9088-48c8d527b591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:45.417967 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64285ad3-8d5e-4b92-9088-48c8d527b591 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:45.429412 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1301/2603] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:28:45.433279 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ebe35-d15e-46ac-afda-2dd4862acd1a admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:45.443638 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ebe35-d15e-46ac-afda-2dd4862acd1a admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:45.448240 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5df4e6-0048-4411-83f5-37e68751388b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:45.448909 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5df4e6-0048-4411-83f5-37e68751388b admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:45.449245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5df4e6-0048-4411-83f5-37e68751388b admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:45.449462 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5df4e6-0048-4411-83f5-37e68751388b admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:45.450365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5df4e6-0048-4411-83f5-37e68751388b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:45.452250 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5df4e6-0048-4411-83f5-37e68751388b admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:45.462912 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1303/2604] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:28:45.467247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64285ad3-8d5e-4b92-9088-48c8d527b591 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:45.478295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64285ad3-8d5e-4b92-9088-48c8d527b591 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:45.484113 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8415008-3294-4d4b-a2f7-ecac2280b9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:45.485027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8415008-3294-4d4b-a2f7-ecac2280b9d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:45.485312 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8415008-3294-4d4b-a2f7-ecac2280b9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:45.485588 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8415008-3294-4d4b-a2f7-ecac2280b9d4 admin admin] PATH_INFO: `/v3/projects/c26ca941f8484976950e87fb474ca7c2/users/c060e8f85d3a41769d454d90ee54e33d/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:45.504502 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8415008-3294-4d4b-a2f7-ecac2280b9d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c26ca941f8484976950e87fb474ca7c2, user_id=c060e8f85d3a41769d454d90ee54e33d, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:45.507140 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8415008-3294-4d4b-a2f7-ecac2280b9d4 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:45.531236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1302/2605] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:28:45 2022] PUT /identity/v3/projects/c26ca941f8484976950e87fb474ca7c2/users/c060e8f85d3a41769d454d90ee54e33d/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:45.538235 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec97da6-5bd7-4e87-b1a6-95b41ca7ef5e None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:45.538527 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec97da6-5bd7-4e87-b1a6-95b41ca7ef5e None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:45.538784 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec97da6-5bd7-4e87-b1a6-95b41ca7ef5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:45.568270 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-6ec97da6-5bd7-4e87-b1a6-95b41ca7ef5e None None] MFA Rules not processed for user `c060e8f85d3a41769d454d90ee54e33d`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:45.569386 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-6ec97da6-5bd7-4e87-b1a6-95b41ca7ef5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:45.634722 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1304/2606] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:28:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:45.639193 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8415008-3294-4d4b-a2f7-ecac2280b9d4 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:45.700841 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8415008-3294-4d4b-a2f7-ecac2280b9d4 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:45.705531 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e11bba-7f93-4f59-8ce0-2f767cd8dfe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:45.706204 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e11bba-7f93-4f59-8ce0-2f767cd8dfe2 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:45.706492 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e11bba-7f93-4f59-8ce0-2f767cd8dfe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:45.706772 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e11bba-7f93-4f59-8ce0-2f767cd8dfe2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:45.707507 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e11bba-7f93-4f59-8ce0-2f767cd8dfe2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:45.709421 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e11bba-7f93-4f59-8ce0-2f767cd8dfe2 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:45.739101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1303/2607] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:28:45 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:28:45.740383 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec97da6-5bd7-4e87-b1a6-95b41ca7ef5e None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:45.752227 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec97da6-5bd7-4e87-b1a6-95b41ca7ef5e None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:45.812225 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5817210-e87f-4d65-87f4-ad03a661cd61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:45.813024 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5817210-e87f-4d65-87f4-ad03a661cd61 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:45.813315 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5817210-e87f-4d65-87f4-ad03a661cd61 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:45.813596 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5817210-e87f-4d65-87f4-ad03a661cd61 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:45.814711 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5817210-e87f-4d65-87f4-ad03a661cd61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:45.816335 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5817210-e87f-4d65-87f4-ad03a661cd61 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:45.837337 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1305/2608] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:28:45 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:28:45.841509 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e11bba-7f93-4f59-8ce0-2f767cd8dfe2 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:45.852285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e11bba-7f93-4f59-8ce0-2f767cd8dfe2 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:45.857792 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619c34c-a239-4667-83a5-55a83a689b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:45.858541 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619c34c-a239-4667-83a5-55a83a689b0f admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:45.858896 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619c34c-a239-4667-83a5-55a83a689b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:45.859271 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619c34c-a239-4667-83a5-55a83a689b0f admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:45.860119 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8619c34c-a239-4667-83a5-55a83a689b0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:45.861809 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8619c34c-a239-4667-83a5-55a83a689b0f admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:45.873586 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1304/2609] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:28:45.878037 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5817210-e87f-4d65-87f4-ad03a661cd61 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:45.888423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5817210-e87f-4d65-87f4-ad03a661cd61 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:45.893207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8365b4-2302-4efb-a276-c5fd40344488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:45.894111 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8365b4-2302-4efb-a276-c5fd40344488 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:45.894400 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8365b4-2302-4efb-a276-c5fd40344488 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:45.894682 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8365b4-2302-4efb-a276-c5fd40344488 admin admin] PATH_INFO: `/v3/projects/1691d59d39814bfeaddea322b466d1ca/users/46b87ebca4b3467abd6f543f8dd048a4/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:45.910099 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8365b4-2302-4efb-a276-c5fd40344488 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1691d59d39814bfeaddea322b466d1ca, user_id=46b87ebca4b3467abd6f543f8dd048a4, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:45.913434 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8365b4-2302-4efb-a276-c5fd40344488 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:45.928921 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1306/2610] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:28:45 2022] PUT /identity/v3/projects/1691d59d39814bfeaddea322b466d1ca/users/46b87ebca4b3467abd6f543f8dd048a4/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:45.933273 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619c34c-a239-4667-83a5-55a83a689b0f admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:45.943859 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619c34c-a239-4667-83a5-55a83a689b0f admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:45.966623 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:46.013194 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845ae21-40f3-430d-a9a8-478bf271d1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:46.013886 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845ae21-40f3-430d-a9a8-478bf271d1ff admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:46.014190 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845ae21-40f3-430d-a9a8-478bf271d1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:46.014469 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845ae21-40f3-430d-a9a8-478bf271d1ff admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:46.015242 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4845ae21-40f3-430d-a9a8-478bf271d1ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:46.017642 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4845ae21-40f3-430d-a9a8-478bf271d1ff admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:46.027964 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1305/2611] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:28:46.032111 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8365b4-2302-4efb-a276-c5fd40344488 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:46.037770 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:46.043793 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8365b4-2302-4efb-a276-c5fd40344488 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:46.051964 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd266df-674e-42c6-b5fd-681e02f87bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:46.052875 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd266df-674e-42c6-b5fd-681e02f87bc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:46.053150 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd266df-674e-42c6-b5fd-681e02f87bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:46.053427 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd266df-674e-42c6-b5fd-681e02f87bc8 admin admin] PATH_INFO: `/v3/projects/1691d59d39814bfeaddea322b466d1ca/users/46b87ebca4b3467abd6f543f8dd048a4/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:46.056998 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd266df-674e-42c6-b5fd-681e02f87bc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1691d59d39814bfeaddea322b466d1ca, user_id=46b87ebca4b3467abd6f543f8dd048a4, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:46.059753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd266df-674e-42c6-b5fd-681e02f87bc8 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:46.071641 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1307/2612] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:28:46 2022] PUT /identity/v3/projects/1691d59d39814bfeaddea322b466d1ca/users/46b87ebca4b3467abd6f543f8dd048a4/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:46.140667 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01786ee-aa04-4f83-a656-12dc22144d68 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:46.141217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01786ee-aa04-4f83-a656-12dc22144d68 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:46.141570 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01786ee-aa04-4f83-a656-12dc22144d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:46.175145 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-c01786ee-aa04-4f83-a656-12dc22144d68 None None] MFA Rules not processed for user `46b87ebca4b3467abd6f543f8dd048a4`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:46.177002 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-c01786ee-aa04-4f83-a656-12dc22144d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:46.260000 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1306/2613] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:28:46 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:46.271546 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd266df-674e-42c6-b5fd-681e02f87bc8 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:46.273462 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-8fd266df-674e-42c6-b5fd-681e02f87bc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:46.335744 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd266df-674e-42c6-b5fd-681e02f87bc8 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:46.339317 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870474db-080e-4519-ac6f-e8d8b63ec0d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:46.339986 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870474db-080e-4519-ac6f-e8d8b63ec0d7 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:46.340273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870474db-080e-4519-ac6f-e8d8b63ec0d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:46.340624 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870474db-080e-4519-ac6f-e8d8b63ec0d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:46.350584 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870474db-080e-4519-ac6f-e8d8b63ec0d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:46.352172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870474db-080e-4519-ac6f-e8d8b63ec0d7 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:46.362475 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1308/2614] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:28:46 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:28:46.424600 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01786ee-aa04-4f83-a656-12dc22144d68 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:46.491260 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01786ee-aa04-4f83-a656-12dc22144d68 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:46.495822 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2141d8-4915-426e-be58-20ad7bbeaeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:46.496774 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2141d8-4915-426e-be58-20ad7bbeaeba admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:46.497056 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2141d8-4915-426e-be58-20ad7bbeaeba admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:46.497333 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2141d8-4915-426e-be58-20ad7bbeaeba admin admin] PATH_INFO: `/v3/users/c060e8f85d3a41769d454d90ee54e33d` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:46.498579 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2141d8-4915-426e-be58-20ad7bbeaeba admin admin] RBAC: Authorizing `identity:delete_user(user_id=c060e8f85d3a41769d454d90ee54e33d)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:46.500564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2141d8-4915-426e-be58-20ad7bbeaeba admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:46.549155 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-9a2141d8-4915-426e-be58-20ad7bbeaeba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c060e8f85d3a41769d454d90ee54e33d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:46.550428 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:46.563999 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-9a2141d8-4915-426e-be58-20ad7bbeaeba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c060e8f85d3a41769d454d90ee54e33d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:46.564809 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-9a2141d8-4915-426e-be58-20ad7bbeaeba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c060e8f85d3a41769d454d90ee54e33d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:46.565515 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-9a2141d8-4915-426e-be58-20ad7bbeaeba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c060e8f85d3a41769d454d90ee54e33d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:46.573115 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-9a2141d8-4915-426e-be58-20ad7bbeaeba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c060e8f85d3a41769d454d90ee54e33d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:46.587435 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1307/2615] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:28:46 2022] DELETE /identity/v3/users/c060e8f85d3a41769d454d90ee54e33d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:46.592000 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6218c-39b7-4b6e-8682-69fe882e57ff None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:46.592585 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6218c-39b7-4b6e-8682-69fe882e57ff None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:46.593063 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6218c-39b7-4b6e-8682-69fe882e57ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:46.612494 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-0ab6218c-39b7-4b6e-8682-69fe882e57ff None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:46.613813 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-0ab6218c-39b7-4b6e-8682-69fe882e57ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:46.672285 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1309/2616] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:28:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:46.683299 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2141d8-4915-426e-be58-20ad7bbeaeba admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:46.684784 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-9a2141d8-4915-426e-be58-20ad7bbeaeba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:46.777024 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2141d8-4915-426e-be58-20ad7bbeaeba admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:46.781022 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b449fc4-c31d-42b6-ae1d-f75e93151c06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:46.781795 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b449fc4-c31d-42b6-ae1d-f75e93151c06 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:46.782148 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b449fc4-c31d-42b6-ae1d-f75e93151c06 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:46.782495 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b449fc4-c31d-42b6-ae1d-f75e93151c06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:46.792241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b449fc4-c31d-42b6-ae1d-f75e93151c06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:46.793792 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b449fc4-c31d-42b6-ae1d-f75e93151c06 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:46.803741 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1308/2617] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:28:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:28:47.151149 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab6218c-39b7-4b6e-8682-69fe882e57ff None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:47.152765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-0ab6218c-39b7-4b6e-8682-69fe882e57ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:47.167633 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab6218c-39b7-4b6e-8682-69fe882e57ff None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:47.172651 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed1c98-9be2-4042-bbef-f131f5b0b242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:47.173533 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed1c98-9be2-4042-bbef-f131f5b0b242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:47.173827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed1c98-9be2-4042-bbef-f131f5b0b242 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:47.174106 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed1c98-9be2-4042-bbef-f131f5b0b242 admin admin] PATH_INFO: `/v3/projects/c26ca941f8484976950e87fb474ca7c2` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:47.175803 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ed1c98-9be2-4042-bbef-f131f5b0b242 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c26ca941f8484976950e87fb474ca7c2)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:47.177763 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ed1c98-9be2-4042-bbef-f131f5b0b242 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:47.238207 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-b5ed1c98-9be2-4042-bbef-f131f5b0b242 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c26ca941f8484976950e87fb474ca7c2', 'request_id': 'req-b5ed1c98-9be2-4042-bbef-f131f5b0b242'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:47.244960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-b5ed1c98-9be2-4042-bbef-f131f5b0b242 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c26ca941f8484976950e87fb474ca7c2', 'request_id': 'req-b5ed1c98-9be2-4042-bbef-f131f5b0b242'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:47.257397 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-b5ed1c98-9be2-4042-bbef-f131f5b0b242 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c26ca941f8484976950e87fb474ca7c2', 'request_id': 'req-b5ed1c98-9be2-4042-bbef-f131f5b0b242'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:47.272803 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-b5ed1c98-9be2-4042-bbef-f131f5b0b242 admin admin] The token cache is being invalidate because project c26ca941f8484976950e87fb474ca7c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:28:47.273098 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-b5ed1c98-9be2-4042-bbef-f131f5b0b242 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:47.273752 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-b5ed1c98-9be2-4042-bbef-f131f5b0b242 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:47.275726 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1310/2618] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:28:47 2022] DELETE /identity/v3/projects/c26ca941f8484976950e87fb474ca7c2 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:47.279817 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b449fc4-c31d-42b6-ae1d-f75e93151c06 service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:47.294389 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b449fc4-c31d-42b6-ae1d-f75e93151c06 service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:47.379916 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e264c-4893-4a2c-9a3b-cc8ee5c53d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:47.380894 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e264c-4893-4a2c-9a3b-cc8ee5c53d2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:47.381259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e264c-4893-4a2c-9a3b-cc8ee5c53d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:47.381617 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e264c-4893-4a2c-9a3b-cc8ee5c53d2b admin admin] PATH_INFO: `/v3/users/46b87ebca4b3467abd6f543f8dd048a4` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:47.382884 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e264c-4893-4a2c-9a3b-cc8ee5c53d2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=46b87ebca4b3467abd6f543f8dd048a4)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:47.384853 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e264c-4893-4a2c-9a3b-cc8ee5c53d2b admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:47.439104 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-c60e264c-4893-4a2c-9a3b-cc8ee5c53d2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46b87ebca4b3467abd6f543f8dd048a4'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:47.452363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-c60e264c-4893-4a2c-9a3b-cc8ee5c53d2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46b87ebca4b3467abd6f543f8dd048a4'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:47.453142 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-c60e264c-4893-4a2c-9a3b-cc8ee5c53d2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46b87ebca4b3467abd6f543f8dd048a4'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:47.453752 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-c60e264c-4893-4a2c-9a3b-cc8ee5c53d2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46b87ebca4b3467abd6f543f8dd048a4'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:47.466412 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:47.474077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-c60e264c-4893-4a2c-9a3b-cc8ee5c53d2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46b87ebca4b3467abd6f543f8dd048a4'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:47.508900 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1309/2619] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:28:47 2022] DELETE /identity/v3/users/46b87ebca4b3467abd6f543f8dd048a4 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:47.800717 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed1c98-9be2-4042-bbef-f131f5b0b242 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:47.802258 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-b5ed1c98-9be2-4042-bbef-f131f5b0b242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:47.818354 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed1c98-9be2-4042-bbef-f131f5b0b242 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:47.832220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:47.873713 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0b919-a961-461a-93d5-7d200503ca4c None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:47.873999 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0b919-a961-461a-93d5-7d200503ca4c None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:47.874287 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0b919-a961-461a-93d5-7d200503ca4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:47.893250 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50625e48-79ee-4863-a0fe-ee4f5e33c247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:47.894139 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50625e48-79ee-4863-a0fe-ee4f5e33c247 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:47.894419 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50625e48-79ee-4863-a0fe-ee4f5e33c247 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:47.894720 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50625e48-79ee-4863-a0fe-ee4f5e33c247 admin admin] PATH_INFO: `/v3/projects/1691d59d39814bfeaddea322b466d1ca` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:47.895920 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50625e48-79ee-4863-a0fe-ee4f5e33c247 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1691d59d39814bfeaddea322b466d1ca)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:47.896127 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-8ce0b919-a961-461a-93d5-7d200503ca4c None None] MFA Rules not processed for user `2f8cf6d908314281a3ddc9d9513d39c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:47.897507 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-8ce0b919-a961-461a-93d5-7d200503ca4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:47.897860 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50625e48-79ee-4863-a0fe-ee4f5e33c247 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:47.958504 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-50625e48-79ee-4863-a0fe-ee4f5e33c247 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1691d59d39814bfeaddea322b466d1ca', 'request_id': 'req-50625e48-79ee-4863-a0fe-ee4f5e33c247'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:47.962969 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1310/2620] 10.222.0.15 () {58 vars in 967 bytes} [Wed Jul 20 15:28:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:28:47.964635 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-50625e48-79ee-4863-a0fe-ee4f5e33c247 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1691d59d39814bfeaddea322b466d1ca', 'request_id': 'req-50625e48-79ee-4863-a0fe-ee4f5e33c247'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:47.973513 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-50625e48-79ee-4863-a0fe-ee4f5e33c247 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1691d59d39814bfeaddea322b466d1ca', 'request_id': 'req-50625e48-79ee-4863-a0fe-ee4f5e33c247'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:47.979347 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce0b919-a961-461a-93d5-7d200503ca4c None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:47.980821 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-8ce0b919-a961-461a-93d5-7d200503ca4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:47.987880 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-50625e48-79ee-4863-a0fe-ee4f5e33c247 admin admin] The token cache is being invalidate because project 1691d59d39814bfeaddea322b466d1ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:28:47.988275 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-50625e48-79ee-4863-a0fe-ee4f5e33c247 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:47.989027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-50625e48-79ee-4863-a0fe-ee4f5e33c247 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:47.995423 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1311/2621] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:28:47 2022] DELETE /identity/v3/projects/1691d59d39814bfeaddea322b466d1ca => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:47.997135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af1e91-93ad-4f3c-95d1-ddd4a218f59d None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:47.997433 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af1e91-93ad-4f3c-95d1-ddd4a218f59d None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:47.997717 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af1e91-93ad-4f3c-95d1-ddd4a218f59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:48.019317 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-32af1e91-93ad-4f3c-95d1-ddd4a218f59d None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:48.020956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-32af1e91-93ad-4f3c-95d1-ddd4a218f59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:48.054492 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce0b919-a961-461a-93d5-7d200503ca4c None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:48.087441 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1312/2622] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:48.095076 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3204a-f03a-4fab-9738-662b6d3d3cfb None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:48.095379 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3204a-f03a-4fab-9738-662b6d3d3cfb None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:48.095655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3204a-f03a-4fab-9738-662b6d3d3cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:48.115069 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-76c3204a-f03a-4fab-9738-662b6d3d3cfb None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:48.116291 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-76c3204a-f03a-4fab-9738-662b6d3d3cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:48.117901 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24dc2e-c4a8-4f48-a43e-234e9e115bdc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f4844988474d7a91133d77906611a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:48.118564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24dc2e-c4a8-4f48-a43e-234e9e115bdc service swift] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:48.118897 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24dc2e-c4a8-4f48-a43e-234e9e115bdc service swift] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:48.119223 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24dc2e-c4a8-4f48-a43e-234e9e115bdc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:48.121138 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-3e24dc2e-c4a8-4f48-a43e-234e9e115bdc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:48.124930 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1313/2623] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:28:48.128832 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3204a-f03a-4fab-9738-662b6d3d3cfb None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:48.137955 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3204a-f03a-4fab-9738-662b6d3d3cfb None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:48.142576 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e02a4-5d8d-4803-ad2b-f1e6fc654bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:48.143702 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e02a4-5d8d-4803-ad2b-f1e6fc654bf3 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:48.144003 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e02a4-5d8d-4803-ad2b-f1e6fc654bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:48.144281 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e02a4-5d8d-4803-ad2b-f1e6fc654bf3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:48.145011 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e02a4-5d8d-4803-ad2b-f1e6fc654bf3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:28:48.145320 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e02a4-5d8d-4803-ad2b-f1e6fc654bf3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:48.146967 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e02a4-5d8d-4803-ad2b-f1e6fc654bf3 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:48.156624 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:48.162408 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1314/2624] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 15:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:28:48.164050 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e02a4-5d8d-4803-ad2b-f1e6fc654bf3 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:48.174863 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e02a4-5d8d-4803-ad2b-f1e6fc654bf3 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:48.180832 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbf60e-2d3d-4c7a-ac30-33d07192391d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:48.181479 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbf60e-2d3d-4c7a-ac30-33d07192391d admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:48.181767 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbf60e-2d3d-4c7a-ac30-33d07192391d admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:48.182043 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbf60e-2d3d-4c7a-ac30-33d07192391d admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:48.182756 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fbf60e-2d3d-4c7a-ac30-33d07192391d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:48.183574 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24dc2e-c4a8-4f48-a43e-234e9e115bdc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:48.184738 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fbf60e-2d3d-4c7a-ac30-33d07192391d admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:48.185208 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24dc2e-c4a8-4f48-a43e-234e9e115bdc service swift] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:48.193368 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1311/2625] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jul 20 15:28:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 15:28:48.214997 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1315/2626] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:28:48 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:28:48.219182 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24dc2e-c4a8-4f48-a43e-234e9e115bdc service swift] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:48.231672 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24dc2e-c4a8-4f48-a43e-234e9e115bdc service swift] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:48.294408 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f4a6da-d04a-4ed2-b61b-06d9c1cc0fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:48.295146 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4a6da-d04a-4ed2-b61b-06d9c1cc0fcb admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:48.295444 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4a6da-d04a-4ed2-b61b-06d9c1cc0fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:48.295861 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4a6da-d04a-4ed2-b61b-06d9c1cc0fcb admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:48.296410 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f4a6da-d04a-4ed2-b61b-06d9c1cc0fcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:48.298448 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f4a6da-d04a-4ed2-b61b-06d9c1cc0fcb admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:48.320676 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1312/2627] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:28:48 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:28:48.324713 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbf60e-2d3d-4c7a-ac30-33d07192391d admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:48.332859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:48.337272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbf60e-2d3d-4c7a-ac30-33d07192391d admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:48.343631 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7085156-2447-490d-a51f-ef289f94ea20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:48.344551 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7085156-2447-490d-a51f-ef289f94ea20 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:48.345203 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7085156-2447-490d-a51f-ef289f94ea20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:48.346060 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7085156-2447-490d-a51f-ef289f94ea20 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:48.347205 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7085156-2447-490d-a51f-ef289f94ea20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:48.349238 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7085156-2447-490d-a51f-ef289f94ea20 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:48.360956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1316/2628] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:28:48.365233 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f4a6da-d04a-4ed2-b61b-06d9c1cc0fcb admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:48.377257 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f4a6da-d04a-4ed2-b61b-06d9c1cc0fcb admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:48.382016 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e38db-dfda-4f8e-80fd-1b43506c3fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:48.382694 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e38db-dfda-4f8e-80fd-1b43506c3fed admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:48.382978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e38db-dfda-4f8e-80fd-1b43506c3fed admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:48.383290 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e38db-dfda-4f8e-80fd-1b43506c3fed admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:48.384016 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e38db-dfda-4f8e-80fd-1b43506c3fed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:48.385639 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e38db-dfda-4f8e-80fd-1b43506c3fed admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:48.396791 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1313/2629] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:28:48.401137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7085156-2447-490d-a51f-ef289f94ea20 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:48.411912 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7085156-2447-490d-a51f-ef289f94ea20 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:48.417324 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254b4a6-c934-4b8b-b12c-0f3979567120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:48.418458 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254b4a6-c934-4b8b-b12c-0f3979567120 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:48.418834 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254b4a6-c934-4b8b-b12c-0f3979567120 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:48.419281 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254b4a6-c934-4b8b-b12c-0f3979567120 admin admin] PATH_INFO: `/v3/projects/c57a575ef9d04890a2b226f33ebc37db/users/bb3a8854de35463c9e1330245b864279/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:48.442065 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7254b4a6-c934-4b8b-b12c-0f3979567120 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c57a575ef9d04890a2b226f33ebc37db, user_id=bb3a8854de35463c9e1330245b864279, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:48.446221 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7254b4a6-c934-4b8b-b12c-0f3979567120 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:48.462210 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1317/2630] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:28:48 2022] PUT /identity/v3/projects/c57a575ef9d04890a2b226f33ebc37db/users/bb3a8854de35463c9e1330245b864279/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:48.473686 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859d6d0-ebf6-4031-a950-ff3a79aa967a None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:48.473929 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859d6d0-ebf6-4031-a950-ff3a79aa967a None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:48.474217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859d6d0-ebf6-4031-a950-ff3a79aa967a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:48.504308 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-0859d6d0-ebf6-4031-a950-ff3a79aa967a None None] MFA Rules not processed for user `bb3a8854de35463c9e1330245b864279`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:48.505564 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-0859d6d0-ebf6-4031-a950-ff3a79aa967a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:48.586109 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1314/2631] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:28:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:48.587542 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254b4a6-c934-4b8b-b12c-0f3979567120 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:48.651849 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254b4a6-c934-4b8b-b12c-0f3979567120 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:48.657622 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0f5f9a-485f-4704-bad6-fab936c172e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:48.658295 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f5f9a-485f-4704-bad6-fab936c172e9 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:48.658584 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f5f9a-485f-4704-bad6-fab936c172e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:48.658903 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f5f9a-485f-4704-bad6-fab936c172e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:48.659673 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0f5f9a-485f-4704-bad6-fab936c172e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:48.661591 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0f5f9a-485f-4704-bad6-fab936c172e9 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:48.689534 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1318/2632] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:28:48 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:28:48.693897 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859d6d0-ebf6-4031-a950-ff3a79aa967a None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:48.705148 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859d6d0-ebf6-4031-a950-ff3a79aa967a None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:48.753232 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:48.772227 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b585c-173d-4cfc-a7c6-32ec1eabbb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:48.772914 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b585c-173d-4cfc-a7c6-32ec1eabbb57 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:48.773199 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b585c-173d-4cfc-a7c6-32ec1eabbb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:48.773478 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b585c-173d-4cfc-a7c6-32ec1eabbb57 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:48.774184 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b585c-173d-4cfc-a7c6-32ec1eabbb57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:48.776345 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b585c-173d-4cfc-a7c6-32ec1eabbb57 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:48.801608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1315/2633] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:28:48 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:28:48.802825 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0f5f9a-485f-4704-bad6-fab936c172e9 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:48.813484 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0f5f9a-485f-4704-bad6-fab936c172e9 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:48.819007 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836673a-2735-42f0-9a5a-610eab08d8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:48.819701 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836673a-2735-42f0-9a5a-610eab08d8cd admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:48.819982 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836673a-2735-42f0-9a5a-610eab08d8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:48.820256 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836673a-2735-42f0-9a5a-610eab08d8cd admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:48.820985 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836673a-2735-42f0-9a5a-610eab08d8cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:48.822594 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836673a-2735-42f0-9a5a-610eab08d8cd admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:48.833843 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1319/2634] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:28:48.837930 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b585c-173d-4cfc-a7c6-32ec1eabbb57 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:48.851603 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b585c-173d-4cfc-a7c6-32ec1eabbb57 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:48.856922 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d859da-d55a-427a-81d0-bc32094b7dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:48.857597 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d859da-d55a-427a-81d0-bc32094b7dde admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:48.857881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d859da-d55a-427a-81d0-bc32094b7dde admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:48.858159 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d859da-d55a-427a-81d0-bc32094b7dde admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:48.858909 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d859da-d55a-427a-81d0-bc32094b7dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:48.860568 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d859da-d55a-427a-81d0-bc32094b7dde admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:48.871137 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1316/2635] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:28:48.875429 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836673a-2735-42f0-9a5a-610eab08d8cd admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:48.885601 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836673a-2735-42f0-9a5a-610eab08d8cd admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:48.890539 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca319dd1-36b0-44af-a74e-9c4477a7f7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:48.891585 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca319dd1-36b0-44af-a74e-9c4477a7f7ac admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:48.891873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca319dd1-36b0-44af-a74e-9c4477a7f7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:48.892151 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca319dd1-36b0-44af-a74e-9c4477a7f7ac admin admin] PATH_INFO: `/v3/projects/85a7ad16b7ea4082a29a25bac123ca08/users/2238683dae7b42b1b916379ddf6b02c6/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:48.906704 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca319dd1-36b0-44af-a74e-9c4477a7f7ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=85a7ad16b7ea4082a29a25bac123ca08, user_id=2238683dae7b42b1b916379ddf6b02c6, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:48.909391 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca319dd1-36b0-44af-a74e-9c4477a7f7ac admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:48.923903 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1320/2636] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:28:48 2022] PUT /identity/v3/projects/85a7ad16b7ea4082a29a25bac123ca08/users/2238683dae7b42b1b916379ddf6b02c6/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:48.935583 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544a0b5-b1f0-49b1-a25d-f9321017f815 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:48.935858 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544a0b5-b1f0-49b1-a25d-f9321017f815 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:48.936162 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544a0b5-b1f0-49b1-a25d-f9321017f815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:48.963962 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-6544a0b5-b1f0-49b1-a25d-f9321017f815 None None] MFA Rules not processed for user `2238683dae7b42b1b916379ddf6b02c6`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:48.965199 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-6544a0b5-b1f0-49b1-a25d-f9321017f815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:49.035532 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1317/2637] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:28:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:49.040109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca319dd1-36b0-44af-a74e-9c4477a7f7ac admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:49.104646 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca319dd1-36b0-44af-a74e-9c4477a7f7ac admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:49.110197 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd1da2-1622-4678-b86a-e0a6881a4122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:49.110865 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd1da2-1622-4678-b86a-e0a6881a4122 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:49.111187 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd1da2-1622-4678-b86a-e0a6881a4122 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:49.111464 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd1da2-1622-4678-b86a-e0a6881a4122 admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:49.112157 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd1da2-1622-4678-b86a-e0a6881a4122 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:49.114068 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd1da2-1622-4678-b86a-e0a6881a4122 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:49.144406 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1321/2638] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:28:49 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:28:49.149683 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6544a0b5-b1f0-49b1-a25d-f9321017f815 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:49.161237 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6544a0b5-b1f0-49b1-a25d-f9321017f815 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:49.220889 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0ae43-fdf4-4024-a313-ae5c96bc075a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:49.221566 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0ae43-fdf4-4024-a313-ae5c96bc075a admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:49.221848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0ae43-fdf4-4024-a313-ae5c96bc075a admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:49.222124 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0ae43-fdf4-4024-a313-ae5c96bc075a admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:49.222819 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0ae43-fdf4-4024-a313-ae5c96bc075a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:49.224834 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0ae43-fdf4-4024-a313-ae5c96bc075a admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:49.246958 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1318/2639] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:28:49 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:28:49.251206 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd1da2-1622-4678-b86a-e0a6881a4122 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:49.258501 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:49.262791 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd1da2-1622-4678-b86a-e0a6881a4122 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:49.268597 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c38afc-a7c3-4ab4-803d-75ada315fd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:49.269304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c38afc-a7c3-4ab4-803d-75ada315fd6e admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:49.269621 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c38afc-a7c3-4ab4-803d-75ada315fd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:49.269990 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c38afc-a7c3-4ab4-803d-75ada315fd6e admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:49.270723 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c38afc-a7c3-4ab4-803d-75ada315fd6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:49.272410 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c38afc-a7c3-4ab4-803d-75ada315fd6e admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:49.283020 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1322/2640] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:28:49.287332 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0ae43-fdf4-4024-a313-ae5c96bc075a admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:49.297844 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0ae43-fdf4-4024-a313-ae5c96bc075a admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:49.306569 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09f349-b30e-406f-97a7-23f20e81ebc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:49.307870 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09f349-b30e-406f-97a7-23f20e81ebc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:49.308219 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09f349-b30e-406f-97a7-23f20e81ebc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:49.308581 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09f349-b30e-406f-97a7-23f20e81ebc0 admin admin] PATH_INFO: `/v3/projects/b26a107b284f46728444fbe0c81cbe00/users/8213d0ccc0fc4dcd9225db479a6e1b7a/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:49.324694 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d09f349-b30e-406f-97a7-23f20e81ebc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b26a107b284f46728444fbe0c81cbe00, user_id=8213d0ccc0fc4dcd9225db479a6e1b7a, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:49.327524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d09f349-b30e-406f-97a7-23f20e81ebc0 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:49.342324 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1319/2641] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:28:49 2022] PUT /identity/v3/projects/b26a107b284f46728444fbe0c81cbe00/users/8213d0ccc0fc4dcd9225db479a6e1b7a/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:49.346366 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c38afc-a7c3-4ab4-803d-75ada315fd6e admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:49.356756 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c38afc-a7c3-4ab4-803d-75ada315fd6e admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:49.424280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa3246-3f5d-4e3a-b710-cf5bcc52901c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:49.425251 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa3246-3f5d-4e3a-b710-cf5bcc52901c admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:49.425643 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa3246-3f5d-4e3a-b710-cf5bcc52901c admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:49.426052 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa3246-3f5d-4e3a-b710-cf5bcc52901c admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:49.427147 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa3246-3f5d-4e3a-b710-cf5bcc52901c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:49.429655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa3246-3f5d-4e3a-b710-cf5bcc52901c admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:49.441678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1323/2642] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:28:49.445992 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09f349-b30e-406f-97a7-23f20e81ebc0 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:49.456742 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09f349-b30e-406f-97a7-23f20e81ebc0 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:49.462089 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928dedb9-a947-4e11-967d-da224541df50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:49.462979 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928dedb9-a947-4e11-967d-da224541df50 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:49.463295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928dedb9-a947-4e11-967d-da224541df50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:49.463575 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928dedb9-a947-4e11-967d-da224541df50 admin admin] PATH_INFO: `/v3/projects/b26a107b284f46728444fbe0c81cbe00/users/8213d0ccc0fc4dcd9225db479a6e1b7a/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:49.465708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928dedb9-a947-4e11-967d-da224541df50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b26a107b284f46728444fbe0c81cbe00, user_id=8213d0ccc0fc4dcd9225db479a6e1b7a, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:49.468377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928dedb9-a947-4e11-967d-da224541df50 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:49.480286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1320/2643] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:28:49 2022] PUT /identity/v3/projects/b26a107b284f46728444fbe0c81cbe00/users/8213d0ccc0fc4dcd9225db479a6e1b7a/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:49.490712 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a9076-fd88-4023-bdaa-979a77f5da85 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:49.491070 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a9076-fd88-4023-bdaa-979a77f5da85 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:49.491361 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a9076-fd88-4023-bdaa-979a77f5da85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:49.519805 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-f52a9076-fd88-4023-bdaa-979a77f5da85 None None] MFA Rules not processed for user `8213d0ccc0fc4dcd9225db479a6e1b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:49.521011 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-f52a9076-fd88-4023-bdaa-979a77f5da85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:49.602554 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1324/2644] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:28:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:49.613039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928dedb9-a947-4e11-967d-da224541df50 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:49.614909 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-928dedb9-a947-4e11-967d-da224541df50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:49.622508 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:49.693855 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928dedb9-a947-4e11-967d-da224541df50 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:49.698849 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f4bf6-69b2-4a3a-9757-2a2d9f847b4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:49.699587 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f4bf6-69b2-4a3a-9757-2a2d9f847b4c service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:49.700159 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f4bf6-69b2-4a3a-9757-2a2d9f847b4c service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:49.700453 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f4bf6-69b2-4a3a-9757-2a2d9f847b4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:49.710584 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f4bf6-69b2-4a3a-9757-2a2d9f847b4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:49.712055 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f4bf6-69b2-4a3a-9757-2a2d9f847b4c service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:49.722827 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1321/2645] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:28:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:28:50.235561 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52a9076-fd88-4023-bdaa-979a77f5da85 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:50.238063 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-f52a9076-fd88-4023-bdaa-979a77f5da85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:50.251407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52a9076-fd88-4023-bdaa-979a77f5da85 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:50.255849 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2caf8e-f54b-40a4-8dac-b1883ed721dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:50.256505 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2caf8e-f54b-40a4-8dac-b1883ed721dd service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:50.256791 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2caf8e-f54b-40a4-8dac-b1883ed721dd service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:50.257064 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2caf8e-f54b-40a4-8dac-b1883ed721dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:50.314131 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2caf8e-f54b-40a4-8dac-b1883ed721dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:50.315613 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2caf8e-f54b-40a4-8dac-b1883ed721dd service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:50.325811 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1325/2646] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:28:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:28:54.386190 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f4bf6-69b2-4a3a-9757-2a2d9f847b4c service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:54.395421 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f4bf6-69b2-4a3a-9757-2a2d9f847b4c service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:54.399321 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd93d0d-247b-4e0c-af19-352e1bc78157 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:54.400065 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd93d0d-247b-4e0c-af19-352e1bc78157 service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:54.400364 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd93d0d-247b-4e0c-af19-352e1bc78157 service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:54.400647 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd93d0d-247b-4e0c-af19-352e1bc78157 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:54.462396 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd93d0d-247b-4e0c-af19-352e1bc78157 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:54.463836 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd93d0d-247b-4e0c-af19-352e1bc78157 service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:54.473234 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1322/2647] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:28:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:28:56.670519 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2caf8e-f54b-40a4-8dac-b1883ed721dd service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:56.692732 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:56.752259 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2caf8e-f54b-40a4-8dac-b1883ed721dd service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:56.757092 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44fbcc-f160-4da5-8143-706a7a9c83af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:56.758012 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44fbcc-f160-4da5-8143-706a7a9c83af admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:56.758300 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44fbcc-f160-4da5-8143-706a7a9c83af admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:56.758615 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44fbcc-f160-4da5-8143-706a7a9c83af admin admin] PATH_INFO: `/v3/users/bb3a8854de35463c9e1330245b864279` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:56.759872 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44fbcc-f160-4da5-8143-706a7a9c83af admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb3a8854de35463c9e1330245b864279)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:56.761883 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44fbcc-f160-4da5-8143-706a7a9c83af admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:56.816699 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1c44fbcc-f160-4da5-8143-706a7a9c83af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb3a8854de35463c9e1330245b864279'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:56.830176 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1c44fbcc-f160-4da5-8143-706a7a9c83af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb3a8854de35463c9e1330245b864279'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:56.831163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1c44fbcc-f160-4da5-8143-706a7a9c83af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb3a8854de35463c9e1330245b864279'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:56.831926 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1c44fbcc-f160-4da5-8143-706a7a9c83af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb3a8854de35463c9e1330245b864279'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:56.840264 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-1c44fbcc-f160-4da5-8143-706a7a9c83af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb3a8854de35463c9e1330245b864279'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:56.858465 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1326/2648] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:28:56 2022] DELETE /identity/v3/users/bb3a8854de35463c9e1330245b864279 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:56.864266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da513e-4eb6-4697-9202-5c1e3044db5a None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:56.864794 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da513e-4eb6-4697-9202-5c1e3044db5a None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:56.864913 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da513e-4eb6-4697-9202-5c1e3044db5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:56.885874 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-01da513e-4eb6-4697-9202-5c1e3044db5a None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:56.887169 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-01da513e-4eb6-4697-9202-5c1e3044db5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:56.955238 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1323/2649] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:28:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:56.965394 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44fbcc-f160-4da5-8143-706a7a9c83af admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:56.967108 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-1c44fbcc-f160-4da5-8143-706a7a9c83af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:57.029935 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44fbcc-f160-4da5-8143-706a7a9c83af admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:57.034432 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d13bad3-9461-468e-90e4-5f9775563756 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:57.035082 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13bad3-9461-468e-90e4-5f9775563756 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:57.035419 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13bad3-9461-468e-90e4-5f9775563756 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:57.035696 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13bad3-9461-468e-90e4-5f9775563756 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:57.045737 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d13bad3-9461-468e-90e4-5f9775563756 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:57.047224 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d13bad3-9461-468e-90e4-5f9775563756 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:57.056985 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1327/2650] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:28:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:28:57.340664 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da513e-4eb6-4697-9202-5c1e3044db5a None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:57.342189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-01da513e-4eb6-4697-9202-5c1e3044db5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:57.354375 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da513e-4eb6-4697-9202-5c1e3044db5a None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:57.359683 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e92e4-d51b-4bc8-9d2b-c1268544deb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:57.360598 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e92e4-d51b-4bc8-9d2b-c1268544deb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:57.360887 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e92e4-d51b-4bc8-9d2b-c1268544deb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:57.361170 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e92e4-d51b-4bc8-9d2b-c1268544deb9 admin admin] PATH_INFO: `/v3/projects/c57a575ef9d04890a2b226f33ebc37db` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:57.362391 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e92e4-d51b-4bc8-9d2b-c1268544deb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c57a575ef9d04890a2b226f33ebc37db)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:57.364373 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e92e4-d51b-4bc8-9d2b-c1268544deb9 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:57.407169 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:57.414008 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-f37e92e4-d51b-4bc8-9d2b-c1268544deb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c57a575ef9d04890a2b226f33ebc37db', 'request_id': 'req-f37e92e4-d51b-4bc8-9d2b-c1268544deb9'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:57.419684 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-f37e92e4-d51b-4bc8-9d2b-c1268544deb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c57a575ef9d04890a2b226f33ebc37db', 'request_id': 'req-f37e92e4-d51b-4bc8-9d2b-c1268544deb9'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:57.425817 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-f37e92e4-d51b-4bc8-9d2b-c1268544deb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c57a575ef9d04890a2b226f33ebc37db', 'request_id': 'req-f37e92e4-d51b-4bc8-9d2b-c1268544deb9'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:57.439375 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-f37e92e4-d51b-4bc8-9d2b-c1268544deb9 admin admin] The token cache is being invalidate because project c57a575ef9d04890a2b226f33ebc37db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:28:57.439670 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-f37e92e4-d51b-4bc8-9d2b-c1268544deb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:57.440346 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-f37e92e4-d51b-4bc8-9d2b-c1268544deb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:57.441902 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1324/2651] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:28:57 2022] DELETE /identity/v3/projects/c57a575ef9d04890a2b226f33ebc37db => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:57.445956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d13bad3-9461-468e-90e4-5f9775563756 service neutron] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:57.455938 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d13bad3-9461-468e-90e4-5f9775563756 service neutron] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:57.496472 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:57.525341 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139a07b-2efb-4b7e-84ba-9d424bcfb985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:57.526537 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139a07b-2efb-4b7e-84ba-9d424bcfb985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:57.526836 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139a07b-2efb-4b7e-84ba-9d424bcfb985 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:57.527178 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139a07b-2efb-4b7e-84ba-9d424bcfb985 admin admin] PATH_INFO: `/v3/users/2238683dae7b42b1b916379ddf6b02c6` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:57.528753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b139a07b-2efb-4b7e-84ba-9d424bcfb985 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2238683dae7b42b1b916379ddf6b02c6)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:57.530701 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b139a07b-2efb-4b7e-84ba-9d424bcfb985 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:57.584954 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-b139a07b-2efb-4b7e-84ba-9d424bcfb985 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2238683dae7b42b1b916379ddf6b02c6'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:57.597181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-b139a07b-2efb-4b7e-84ba-9d424bcfb985 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2238683dae7b42b1b916379ddf6b02c6'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:57.597908 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-b139a07b-2efb-4b7e-84ba-9d424bcfb985 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2238683dae7b42b1b916379ddf6b02c6'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:57.598559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-b139a07b-2efb-4b7e-84ba-9d424bcfb985 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2238683dae7b42b1b916379ddf6b02c6'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:57.605641 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-b139a07b-2efb-4b7e-84ba-9d424bcfb985 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2238683dae7b42b1b916379ddf6b02c6'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:57.619344 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1328/2652] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:28:57 2022] DELETE /identity/v3/users/2238683dae7b42b1b916379ddf6b02c6 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:57.907530 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e92e4-d51b-4bc8-9d2b-c1268544deb9 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:57.909710 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-f37e92e4-d51b-4bc8-9d2b-c1268544deb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:57.926274 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e92e4-d51b-4bc8-9d2b-c1268544deb9 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:57.985648 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a5fb3-b9d3-48c1-a967-42b1728f98fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:57.986526 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a5fb3-b9d3-48c1-a967-42b1728f98fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:57.986811 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a5fb3-b9d3-48c1-a967-42b1728f98fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:57.987095 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a5fb3-b9d3-48c1-a967-42b1728f98fe admin admin] PATH_INFO: `/v3/projects/85a7ad16b7ea4082a29a25bac123ca08` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:57.988303 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a5fb3-b9d3-48c1-a967-42b1728f98fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=85a7ad16b7ea4082a29a25bac123ca08)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:57.990239 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a5fb3-b9d3-48c1-a967-42b1728f98fe admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:58.038711 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ce8a5fb3-b9d3-48c1-a967-42b1728f98fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85a7ad16b7ea4082a29a25bac123ca08', 'request_id': 'req-ce8a5fb3-b9d3-48c1-a967-42b1728f98fe'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:58.044304 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ce8a5fb3-b9d3-48c1-a967-42b1728f98fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85a7ad16b7ea4082a29a25bac123ca08', 'request_id': 'req-ce8a5fb3-b9d3-48c1-a967-42b1728f98fe'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:58.050354 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ce8a5fb3-b9d3-48c1-a967-42b1728f98fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85a7ad16b7ea4082a29a25bac123ca08', 'request_id': 'req-ce8a5fb3-b9d3-48c1-a967-42b1728f98fe'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:58.064272 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ce8a5fb3-b9d3-48c1-a967-42b1728f98fe admin admin] The token cache is being invalidate because project 85a7ad16b7ea4082a29a25bac123ca08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:28:58.064584 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ce8a5fb3-b9d3-48c1-a967-42b1728f98fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:58.065291 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ce8a5fb3-b9d3-48c1-a967-42b1728f98fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:58.066971 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1325/2653] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:28:57 2022] DELETE /identity/v3/projects/85a7ad16b7ea4082a29a25bac123ca08 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:58.071099 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139a07b-2efb-4b7e-84ba-9d424bcfb985 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:58.072656 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-b139a07b-2efb-4b7e-84ba-9d424bcfb985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:58.086471 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139a07b-2efb-4b7e-84ba-9d424bcfb985 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:58.147217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc0fe6-affa-4de1-a50d-8eb7e7bf158b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:58.148025 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc0fe6-affa-4de1-a50d-8eb7e7bf158b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:58.148309 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc0fe6-affa-4de1-a50d-8eb7e7bf158b admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:58.148585 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc0fe6-affa-4de1-a50d-8eb7e7bf158b admin admin] PATH_INFO: `/v3/users/8213d0ccc0fc4dcd9225db479a6e1b7a` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:58.149844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc0fe6-affa-4de1-a50d-8eb7e7bf158b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8213d0ccc0fc4dcd9225db479a6e1b7a)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:58.151825 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc0fe6-affa-4de1-a50d-8eb7e7bf158b admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:58.203510 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-23bc0fe6-affa-4de1-a50d-8eb7e7bf158b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8213d0ccc0fc4dcd9225db479a6e1b7a'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:58.211020 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:58.217916 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-23bc0fe6-affa-4de1-a50d-8eb7e7bf158b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8213d0ccc0fc4dcd9225db479a6e1b7a'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:58.218768 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-23bc0fe6-affa-4de1-a50d-8eb7e7bf158b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8213d0ccc0fc4dcd9225db479a6e1b7a'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:58.219883 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-23bc0fe6-affa-4de1-a50d-8eb7e7bf158b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8213d0ccc0fc4dcd9225db479a6e1b7a'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:58.227370 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-23bc0fe6-affa-4de1-a50d-8eb7e7bf158b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8213d0ccc0fc4dcd9225db479a6e1b7a'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:58.242747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1329/2654] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:28:58 2022] DELETE /identity/v3/users/8213d0ccc0fc4dcd9225db479a6e1b7a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:58.532841 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a5fb3-b9d3-48c1-a967-42b1728f98fe admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:58.534337 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-ce8a5fb3-b9d3-48c1-a967-42b1728f98fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:58.547669 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a5fb3-b9d3-48c1-a967-42b1728f98fe admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:58.622849 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c42e99-40e0-446b-b880-b31e2b98470a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:58.623757 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c42e99-40e0-446b-b880-b31e2b98470a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:58.624042 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c42e99-40e0-446b-b880-b31e2b98470a admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:58.624320 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c42e99-40e0-446b-b880-b31e2b98470a admin admin] PATH_INFO: `/v3/projects/b26a107b284f46728444fbe0c81cbe00` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:58.625565 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c42e99-40e0-446b-b880-b31e2b98470a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b26a107b284f46728444fbe0c81cbe00)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:58.627527 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c42e99-40e0-446b-b880-b31e2b98470a admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:58.629087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:58.678894 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-a6c42e99-40e0-446b-b880-b31e2b98470a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b26a107b284f46728444fbe0c81cbe00', 'request_id': 'req-a6c42e99-40e0-446b-b880-b31e2b98470a'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:58.684780 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-a6c42e99-40e0-446b-b880-b31e2b98470a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b26a107b284f46728444fbe0c81cbe00', 'request_id': 'req-a6c42e99-40e0-446b-b880-b31e2b98470a'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:58.691135 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-a6c42e99-40e0-446b-b880-b31e2b98470a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b26a107b284f46728444fbe0c81cbe00', 'request_id': 'req-a6c42e99-40e0-446b-b880-b31e2b98470a'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:58.704537 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-a6c42e99-40e0-446b-b880-b31e2b98470a admin admin] The token cache is being invalidate because project b26a107b284f46728444fbe0c81cbe00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:28:58.704819 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-a6c42e99-40e0-446b-b880-b31e2b98470a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:58.705432 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-a6c42e99-40e0-446b-b880-b31e2b98470a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:28:58.706964 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1326/2655] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:28:58 2022] DELETE /identity/v3/projects/b26a107b284f46728444fbe0c81cbe00 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:58.713053 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89e38d-0103-4ada-a657-c479d49cc53c None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:58.713346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89e38d-0103-4ada-a657-c479d49cc53c None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:58.713623 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89e38d-0103-4ada-a657-c479d49cc53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:58.733051 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-9a89e38d-0103-4ada-a657-c479d49cc53c None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:58.734252 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-9a89e38d-0103-4ada-a657-c479d49cc53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:58.798528 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1330/2656] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:28:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:58.808927 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aed980-2382-4914-9bb2-991df74aca2b None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:58.809302 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aed980-2382-4914-9bb2-991df74aca2b None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:58.809912 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aed980-2382-4914-9bb2-991df74aca2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:58.833699 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-38aed980-2382-4914-9bb2-991df74aca2b None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:58.834924 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-38aed980-2382-4914-9bb2-991df74aca2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:58.844359 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1327/2657] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:28:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:58.848939 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a89e38d-0103-4ada-a657-c479d49cc53c None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:58.859791 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a89e38d-0103-4ada-a657-c479d49cc53c None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:58.864990 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03812c6-2c81-4575-86ce-eeec4e6f4e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:58.865651 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03812c6-2c81-4575-86ce-eeec4e6f4e53 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:58.865935 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03812c6-2c81-4575-86ce-eeec4e6f4e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:58.866204 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03812c6-2c81-4575-86ce-eeec4e6f4e53 admin admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:58.866926 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03812c6-2c81-4575-86ce-eeec4e6f4e53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:28:58.867318 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03812c6-2c81-4575-86ce-eeec4e6f4e53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:58.868895 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03812c6-2c81-4575-86ce-eeec4e6f4e53 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:58.881243 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1331/2658] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 15:28:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:28:58.886039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aed980-2382-4914-9bb2-991df74aca2b None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:58.896018 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aed980-2382-4914-9bb2-991df74aca2b None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:58.902626 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de79b11-5123-4c11-a880-dc368dbd54be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:58.903369 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de79b11-5123-4c11-a880-dc368dbd54be admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:58.903653 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de79b11-5123-4c11-a880-dc368dbd54be admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:58.903931 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de79b11-5123-4c11-a880-dc368dbd54be admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:58.904605 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de79b11-5123-4c11-a880-dc368dbd54be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:58.906605 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de79b11-5123-4c11-a880-dc368dbd54be admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:58.935533 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1328/2659] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:28:58 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:28:58.939823 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03812c6-2c81-4575-86ce-eeec4e6f4e53 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:58.950115 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03812c6-2c81-4575-86ce-eeec4e6f4e53 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:59.011146 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e275389-61d7-4b45-90d6-80a01ccf2c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:59.011829 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e275389-61d7-4b45-90d6-80a01ccf2c5b admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:59.012110 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e275389-61d7-4b45-90d6-80a01ccf2c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:59.012403 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e275389-61d7-4b45-90d6-80a01ccf2c5b admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:59.013124 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e275389-61d7-4b45-90d6-80a01ccf2c5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:59.015034 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e275389-61d7-4b45-90d6-80a01ccf2c5b admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:59.035579 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1332/2660] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:28:58 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:28:59.039932 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de79b11-5123-4c11-a880-dc368dbd54be admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:59.052559 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de79b11-5123-4c11-a880-dc368dbd54be admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:59.057254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1aeeb-afd9-4b16-8347-0d0286567841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:59.057960 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1aeeb-afd9-4b16-8347-0d0286567841 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:59.058246 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1aeeb-afd9-4b16-8347-0d0286567841 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:59.058543 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1aeeb-afd9-4b16-8347-0d0286567841 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:59.059371 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1aeeb-afd9-4b16-8347-0d0286567841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:59.060989 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1aeeb-afd9-4b16-8347-0d0286567841 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:59.071718 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1329/2661] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:28:59.075900 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e275389-61d7-4b45-90d6-80a01ccf2c5b admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:59.086870 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e275389-61d7-4b45-90d6-80a01ccf2c5b admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:59.092848 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f10e93c-259b-4bca-b562-9f74d0c5e740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:59.093540 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10e93c-259b-4bca-b562-9f74d0c5e740 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:59.093826 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10e93c-259b-4bca-b562-9f74d0c5e740 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:59.094097 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10e93c-259b-4bca-b562-9f74d0c5e740 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:59.094804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f10e93c-259b-4bca-b562-9f74d0c5e740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:59.096430 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f10e93c-259b-4bca-b562-9f74d0c5e740 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:59.107356 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1333/2662] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:28:59.111523 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1aeeb-afd9-4b16-8347-0d0286567841 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:59.122115 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1aeeb-afd9-4b16-8347-0d0286567841 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:59.127952 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1106144-fb50-4c25-adb7-7f358e707c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:59.128838 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1106144-fb50-4c25-adb7-7f358e707c13 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:59.129118 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1106144-fb50-4c25-adb7-7f358e707c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:59.129393 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1106144-fb50-4c25-adb7-7f358e707c13 admin admin] PATH_INFO: `/v3/projects/e3664821804a4cb388d3fe53d270a45f/users/7f2bb03b6f6147e899d121fdf98f9408/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:59.146571 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1106144-fb50-4c25-adb7-7f358e707c13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3664821804a4cb388d3fe53d270a45f, user_id=7f2bb03b6f6147e899d121fdf98f9408, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:59.150764 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1106144-fb50-4c25-adb7-7f358e707c13 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:59.167152 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1330/2663] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:28:59 2022] PUT /identity/v3/projects/e3664821804a4cb388d3fe53d270a45f/users/7f2bb03b6f6147e899d121fdf98f9408/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:59.178365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb32f6-54e7-4599-b207-08bcd61e3595 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:59.178686 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb32f6-54e7-4599-b207-08bcd61e3595 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:59.178965 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb32f6-54e7-4599-b207-08bcd61e3595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:59.207029 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-0beb32f6-54e7-4599-b207-08bcd61e3595 None None] MFA Rules not processed for user `7f2bb03b6f6147e899d121fdf98f9408`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:59.208238 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-0beb32f6-54e7-4599-b207-08bcd61e3595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:59.237832 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:59.279220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1334/2664] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:28:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:59.291451 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1106144-fb50-4c25-adb7-7f358e707c13 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:59.293747 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-b1106144-fb50-4c25-adb7-7f358e707c13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:59.359886 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1106144-fb50-4c25-adb7-7f358e707c13 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:59.364651 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02abba4-60c8-4aaf-8943-5c4840fcc7bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:59.365305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02abba4-60c8-4aaf-8943-5c4840fcc7bb service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:59.365588 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02abba4-60c8-4aaf-8943-5c4840fcc7bb service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:59.365862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02abba4-60c8-4aaf-8943-5c4840fcc7bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:59.375065 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02abba4-60c8-4aaf-8943-5c4840fcc7bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:59.376559 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02abba4-60c8-4aaf-8943-5c4840fcc7bb service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:59.385529 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1331/2665] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:28:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:29:01.962188 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb32f6-54e7-4599-b207-08bcd61e3595 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:29:02.038153 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb32f6-54e7-4599-b207-08bcd61e3595 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:29:02.043172 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7749c0-e33b-4ef8-8ab8-2b718f334a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:29:02.044086 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7749c0-e33b-4ef8-8ab8-2b718f334a4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:02.044370 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7749c0-e33b-4ef8-8ab8-2b718f334a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:02.044645 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7749c0-e33b-4ef8-8ab8-2b718f334a4b admin admin] PATH_INFO: `/v3/users/7f2bb03b6f6147e899d121fdf98f9408` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:02.045855 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7749c0-e33b-4ef8-8ab8-2b718f334a4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f2bb03b6f6147e899d121fdf98f9408)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:29:02.047828 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7749c0-e33b-4ef8-8ab8-2b718f334a4b admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:29:02.102852 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-8c7749c0-e33b-4ef8-8ab8-2b718f334a4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f2bb03b6f6147e899d121fdf98f9408'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:29:02.115748 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-8c7749c0-e33b-4ef8-8ab8-2b718f334a4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f2bb03b6f6147e899d121fdf98f9408'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:29:02.116435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-8c7749c0-e33b-4ef8-8ab8-2b718f334a4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f2bb03b6f6147e899d121fdf98f9408'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:29:02.117071 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-8c7749c0-e33b-4ef8-8ab8-2b718f334a4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f2bb03b6f6147e899d121fdf98f9408'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:29:02.124213 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-8c7749c0-e33b-4ef8-8ab8-2b718f334a4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f2bb03b6f6147e899d121fdf98f9408'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:29:02.140295 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1335/2666] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:29:01 2022] DELETE /identity/v3/users/7f2bb03b6f6147e899d121fdf98f9408 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:29:02.144978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c058d9-b765-4786-8d1a-6f2b0121a527 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:02.145498 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c058d9-b765-4786-8d1a-6f2b0121a527 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:02.145615 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c058d9-b765-4786-8d1a-6f2b0121a527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:02.163573 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-f1c058d9-b765-4786-8d1a-6f2b0121a527 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:02.164858 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-f1c058d9-b765-4786-8d1a-6f2b0121a527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:02.191709 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:02.229536 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1332/2667] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:29:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:29:02.239762 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7749c0-e33b-4ef8-8ab8-2b718f334a4b admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:29:02.241202 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-8c7749c0-e33b-4ef8-8ab8-2b718f334a4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:02.294159 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:02.298776 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7749c0-e33b-4ef8-8ab8-2b718f334a4b admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:29:02.303448 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599399c4-7871-499e-9e91-ccbc03686456 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:29:02.304175 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599399c4-7871-499e-9e91-ccbc03686456 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:02.304920 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599399c4-7871-499e-9e91-ccbc03686456 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:02.305195 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599399c4-7871-499e-9e91-ccbc03686456 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:02.315439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599399c4-7871-499e-9e91-ccbc03686456 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:29:02.316967 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599399c4-7871-499e-9e91-ccbc03686456 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:29:02.327359 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1336/2668] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:29:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:29:02.609550 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c058d9-b765-4786-8d1a-6f2b0121a527 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:29:02.611254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-f1c058d9-b765-4786-8d1a-6f2b0121a527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:02.628473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c058d9-b765-4786-8d1a-6f2b0121a527 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:29:02.633627 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd0c84-acb0-48bc-95dc-fcbd5ab1175b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:29:02.634486 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd0c84-acb0-48bc-95dc-fcbd5ab1175b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:02.634760 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd0c84-acb0-48bc-95dc-fcbd5ab1175b admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:02.635090 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd0c84-acb0-48bc-95dc-fcbd5ab1175b admin admin] PATH_INFO: `/v3/projects/e3664821804a4cb388d3fe53d270a45f` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:02.636368 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dd0c84-acb0-48bc-95dc-fcbd5ab1175b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3664821804a4cb388d3fe53d270a45f)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:29:02.638289 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dd0c84-acb0-48bc-95dc-fcbd5ab1175b admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:29:02.695303 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-24dd0c84-acb0-48bc-95dc-fcbd5ab1175b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3664821804a4cb388d3fe53d270a45f', 'request_id': 'req-24dd0c84-acb0-48bc-95dc-fcbd5ab1175b'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:29:02.702135 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-24dd0c84-acb0-48bc-95dc-fcbd5ab1175b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3664821804a4cb388d3fe53d270a45f', 'request_id': 'req-24dd0c84-acb0-48bc-95dc-fcbd5ab1175b'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:29:02.709284 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-24dd0c84-acb0-48bc-95dc-fcbd5ab1175b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3664821804a4cb388d3fe53d270a45f', 'request_id': 'req-24dd0c84-acb0-48bc-95dc-fcbd5ab1175b'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:29:02.724305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-24dd0c84-acb0-48bc-95dc-fcbd5ab1175b admin admin] The token cache is being invalidate because project e3664821804a4cb388d3fe53d270a45f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:29:02.724529 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-24dd0c84-acb0-48bc-95dc-fcbd5ab1175b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:29:02.725151 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-24dd0c84-acb0-48bc-95dc-fcbd5ab1175b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:29:02.726842 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1333/2669] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:29:02 2022] DELETE /identity/v3/projects/e3664821804a4cb388d3fe53d270a45f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:29:12.322825 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=4 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:12.721125 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=41,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:13.425737 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599399c4-7871-499e-9e91-ccbc03686456 service neutron] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:29:13.427945 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-599399c4-7871-499e-9e91-ccbc03686456 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:13.455676 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599399c4-7871-499e-9e91-ccbc03686456 service neutron] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:29:13.517214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf2a210-6fd5-432d-8108-305eb67f5598 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:29:13.517884 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2a210-6fd5-432d-8108-305eb67f5598 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:13.518173 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2a210-6fd5-432d-8108-305eb67f5598 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:13.518454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2a210-6fd5-432d-8108-305eb67f5598 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:13.520366 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-6bf2a210-6fd5-432d-8108-305eb67f5598 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:13.537781 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf2a210-6fd5-432d-8108-305eb67f5598 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:29:13.539287 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf2a210-6fd5-432d-8108-305eb67f5598 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:29:13.548098 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1337/2670] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:29:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:29:23.544127 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=30 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:04.214287 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd0c84-acb0-48bc-95dc-fcbd5ab1175b admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:04.215766 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-24dd0c84-acb0-48bc-95dc-fcbd5ab1175b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:04.229507 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd0c84-acb0-48bc-95dc-fcbd5ab1175b admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:04.234155 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc6f10-7874-4a15-bab0-a97a98239756 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:04.234817 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc6f10-7874-4a15-bab0-a97a98239756 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:04.235143 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc6f10-7874-4a15-bab0-a97a98239756 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:04.235427 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc6f10-7874-4a15-bab0-a97a98239756 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:04.237437 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-63bc6f10-7874-4a15-bab0-a97a98239756 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:04.314757 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc6f10-7874-4a15-bab0-a97a98239756 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:04.316841 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc6f10-7874-4a15-bab0-a97a98239756 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:04.329311 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1334/2671] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:07.630389 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf2a210-6fd5-432d-8108-305eb67f5598 service neutron] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:07.632200 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-6bf2a210-6fd5-432d-8108-305eb67f5598 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:07.648858 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf2a210-6fd5-432d-8108-305eb67f5598 service neutron] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:07.653792 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51b819-03ed-46b7-80f3-11de30c6e9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:07.654695 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51b819-03ed-46b7-80f3-11de30c6e9ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:07.655025 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51b819-03ed-46b7-80f3-11de30c6e9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:07.655310 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51b819-03ed-46b7-80f3-11de30c6e9ad admin admin] PATH_INFO: `/v3/users/ee5a234904e64eb9b2faad85909d0507` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:07.656575 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc51b819-03ed-46b7-80f3-11de30c6e9ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee5a234904e64eb9b2faad85909d0507)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:07.658628 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc51b819-03ed-46b7-80f3-11de30c6e9ad admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:07.716405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-dc51b819-03ed-46b7-80f3-11de30c6e9ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee5a234904e64eb9b2faad85909d0507'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:07.729365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-dc51b819-03ed-46b7-80f3-11de30c6e9ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee5a234904e64eb9b2faad85909d0507'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:07.730984 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-dc51b819-03ed-46b7-80f3-11de30c6e9ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee5a234904e64eb9b2faad85909d0507'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:07.730984 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-dc51b819-03ed-46b7-80f3-11de30c6e9ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee5a234904e64eb9b2faad85909d0507'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:07.738345 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-dc51b819-03ed-46b7-80f3-11de30c6e9ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee5a234904e64eb9b2faad85909d0507'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:07.753351 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1338/2672] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:30:07 2022] DELETE /identity/v3/users/ee5a234904e64eb9b2faad85909d0507 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:30:07.941481 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc6f10-7874-4a15-bab0-a97a98239756 service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:07.953704 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc6f10-7874-4a15-bab0-a97a98239756 service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:08.016443 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c82c6-e432-4973-91fa-335d7ddacd4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:08.017324 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c82c6-e432-4973-91fa-335d7ddacd4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:08.017613 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c82c6-e432-4973-91fa-335d7ddacd4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:08.017919 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c82c6-e432-4973-91fa-335d7ddacd4c admin admin] PATH_INFO: `/v3/projects/7db1a5ab36714ee2990b9c6c10ba0aea` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:08.019206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241c82c6-e432-4973-91fa-335d7ddacd4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7db1a5ab36714ee2990b9c6c10ba0aea)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:08.021211 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241c82c6-e432-4973-91fa-335d7ddacd4c admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:08.068426 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-241c82c6-e432-4973-91fa-335d7ddacd4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7db1a5ab36714ee2990b9c6c10ba0aea', 'request_id': 'req-241c82c6-e432-4973-91fa-335d7ddacd4c'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:08.074340 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-241c82c6-e432-4973-91fa-335d7ddacd4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7db1a5ab36714ee2990b9c6c10ba0aea', 'request_id': 'req-241c82c6-e432-4973-91fa-335d7ddacd4c'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:08.080788 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-241c82c6-e432-4973-91fa-335d7ddacd4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7db1a5ab36714ee2990b9c6c10ba0aea', 'request_id': 'req-241c82c6-e432-4973-91fa-335d7ddacd4c'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:08.093654 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-241c82c6-e432-4973-91fa-335d7ddacd4c admin admin] The token cache is being invalidate because project 7db1a5ab36714ee2990b9c6c10ba0aea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:30:08.093946 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-241c82c6-e432-4973-91fa-335d7ddacd4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:08.094600 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-241c82c6-e432-4973-91fa-335d7ddacd4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:08.096224 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1335/2673] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:30:07 2022] DELETE /identity/v3/projects/7db1a5ab36714ee2990b9c6c10ba0aea => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:08.101568 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec3a3cf-0361-422d-9ec6-ca7764cd3585 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:08.101863 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec3a3cf-0361-422d-9ec6-ca7764cd3585 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:08.102146 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec3a3cf-0361-422d-9ec6-ca7764cd3585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:08.122285 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-fec3a3cf-0361-422d-9ec6-ca7764cd3585 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:08.123549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-fec3a3cf-0361-422d-9ec6-ca7764cd3585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:08.191441 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1339/2674] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:30:08.199702 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38986cdf-ea8a-440c-9b09-783a58243f40 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:08.199968 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38986cdf-ea8a-440c-9b09-783a58243f40 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:08.200287 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38986cdf-ea8a-440c-9b09-783a58243f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:08.226805 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-38986cdf-ea8a-440c-9b09-783a58243f40 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:08.228092 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-38986cdf-ea8a-440c-9b09-783a58243f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:08.236670 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1336/2675] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:08.241277 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec3a3cf-0361-422d-9ec6-ca7764cd3585 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:08.251812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec3a3cf-0361-422d-9ec6-ca7764cd3585 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:08.256799 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c470a09-448a-4c89-b0c1-3e00ef453201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:08.257478 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c470a09-448a-4c89-b0c1-3e00ef453201 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:08.257764 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c470a09-448a-4c89-b0c1-3e00ef453201 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:08.258044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c470a09-448a-4c89-b0c1-3e00ef453201 admin admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:08.258790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c470a09-448a-4c89-b0c1-3e00ef453201 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:30:08.259137 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c470a09-448a-4c89-b0c1-3e00ef453201 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:08.260887 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c470a09-448a-4c89-b0c1-3e00ef453201 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:08.273967 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1340/2676] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 15:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:30:08.279973 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38986cdf-ea8a-440c-9b09-783a58243f40 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:08.294038 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38986cdf-ea8a-440c-9b09-783a58243f40 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:08.300324 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf16a11-2b76-468d-9453-b252ce495079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:08.301022 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf16a11-2b76-468d-9453-b252ce495079 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:08.301330 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf16a11-2b76-468d-9453-b252ce495079 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:08.301614 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf16a11-2b76-468d-9453-b252ce495079 admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:08.302305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf16a11-2b76-468d-9453-b252ce495079 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:08.304334 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf16a11-2b76-468d-9453-b252ce495079 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:08.320671 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:08.337734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1337/2677] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:30:08 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:30:08.342309 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c470a09-448a-4c89-b0c1-3e00ef453201 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:08.353204 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c470a09-448a-4c89-b0c1-3e00ef453201 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:08.423181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9584b60-199c-43d5-9152-3a4117b4095d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:08.423908 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9584b60-199c-43d5-9152-3a4117b4095d admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:08.424194 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9584b60-199c-43d5-9152-3a4117b4095d admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:08.424473 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9584b60-199c-43d5-9152-3a4117b4095d admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:08.425194 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9584b60-199c-43d5-9152-3a4117b4095d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:08.427208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9584b60-199c-43d5-9152-3a4117b4095d admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:08.434287 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:08.455670 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1341/2678] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:30:08 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:30:08.456896 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf16a11-2b76-468d-9453-b252ce495079 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:08.468151 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf16a11-2b76-468d-9453-b252ce495079 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:08.473255 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69c2b84-3ee8-489f-9d95-cbc4cb91e797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:08.473943 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c2b84-3ee8-489f-9d95-cbc4cb91e797 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:08.474246 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c2b84-3ee8-489f-9d95-cbc4cb91e797 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:08.474545 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c2b84-3ee8-489f-9d95-cbc4cb91e797 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:08.475790 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69c2b84-3ee8-489f-9d95-cbc4cb91e797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:08.477573 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69c2b84-3ee8-489f-9d95-cbc4cb91e797 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:08.494518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1338/2679] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:30:08.498693 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9584b60-199c-43d5-9152-3a4117b4095d admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:08.509053 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9584b60-199c-43d5-9152-3a4117b4095d admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:08.513927 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b4ca1-087d-4807-9f6f-45f3a179088e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:08.514601 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b4ca1-087d-4807-9f6f-45f3a179088e admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:08.514882 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b4ca1-087d-4807-9f6f-45f3a179088e admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:08.515198 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b4ca1-087d-4807-9f6f-45f3a179088e admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:08.515956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b4ca1-087d-4807-9f6f-45f3a179088e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:08.517735 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b4ca1-087d-4807-9f6f-45f3a179088e admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:08.530188 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1342/2680] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:30:08.534542 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69c2b84-3ee8-489f-9d95-cbc4cb91e797 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:08.545922 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69c2b84-3ee8-489f-9d95-cbc4cb91e797 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:08.551022 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b8869-345e-49ab-8df7-f3b999b87d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:08.551938 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b8869-345e-49ab-8df7-f3b999b87d15 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:08.552218 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b8869-345e-49ab-8df7-f3b999b87d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:08.552491 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b8869-345e-49ab-8df7-f3b999b87d15 admin admin] PATH_INFO: `/v3/projects/a2891c3f9d95446b951eb7abe7624f6d/users/c5516a90c978481593de115b69a42a2b/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:08.567700 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060b8869-345e-49ab-8df7-f3b999b87d15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2891c3f9d95446b951eb7abe7624f6d, user_id=c5516a90c978481593de115b69a42a2b, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:08.570527 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060b8869-345e-49ab-8df7-f3b999b87d15 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:08.594088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1339/2681] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:30:08 2022] PUT /identity/v3/projects/a2891c3f9d95446b951eb7abe7624f6d/users/c5516a90c978481593de115b69a42a2b/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:08.606156 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0762e-9b07-49a9-bc12-0431be2e38de None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:08.606445 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0762e-9b07-49a9-bc12-0431be2e38de None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:08.606724 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0762e-9b07-49a9-bc12-0431be2e38de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:08.635522 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-83d0762e-9b07-49a9-bc12-0431be2e38de None None] MFA Rules not processed for user `c5516a90c978481593de115b69a42a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:08.636790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-83d0762e-9b07-49a9-bc12-0431be2e38de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:08.718473 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1343/2682] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:08.732866 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b8869-345e-49ab-8df7-f3b999b87d15 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:08.734801 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-060b8869-345e-49ab-8df7-f3b999b87d15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:08.829710 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b8869-345e-49ab-8df7-f3b999b87d15 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:08.837528 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a675cf-08d8-4f3f-8aec-dfdae420db14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:08.838342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a675cf-08d8-4f3f-8aec-dfdae420db14 service nova] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:08.838632 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a675cf-08d8-4f3f-8aec-dfdae420db14 service nova] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:08.839082 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a675cf-08d8-4f3f-8aec-dfdae420db14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:08.847996 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a675cf-08d8-4f3f-8aec-dfdae420db14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:08.849660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a675cf-08d8-4f3f-8aec-dfdae420db14 service nova] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:08.859260 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1340/2683] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:11.639202 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d0762e-9b07-49a9-bc12-0431be2e38de None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:11.705058 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d0762e-9b07-49a9-bc12-0431be2e38de None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:11.710365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50d06e-d1bb-4157-8745-3d750123de1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:11.711289 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50d06e-d1bb-4157-8745-3d750123de1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:11.711573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50d06e-d1bb-4157-8745-3d750123de1b admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:11.711871 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50d06e-d1bb-4157-8745-3d750123de1b admin admin] PATH_INFO: `/v3/users/c5516a90c978481593de115b69a42a2b` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:11.713222 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de50d06e-d1bb-4157-8745-3d750123de1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5516a90c978481593de115b69a42a2b)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:11.715335 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de50d06e-d1bb-4157-8745-3d750123de1b admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:11.766329 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-de50d06e-d1bb-4157-8745-3d750123de1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5516a90c978481593de115b69a42a2b'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:11.779336 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-de50d06e-d1bb-4157-8745-3d750123de1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5516a90c978481593de115b69a42a2b'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:11.780110 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-de50d06e-d1bb-4157-8745-3d750123de1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5516a90c978481593de115b69a42a2b'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:11.780882 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-de50d06e-d1bb-4157-8745-3d750123de1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5516a90c978481593de115b69a42a2b'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:11.785718 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:DELETE=9 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:11.790149 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-de50d06e-d1bb-4157-8745-3d750123de1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5516a90c978481593de115b69a42a2b'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:11.811143 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1344/2684] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:30:11 2022] DELETE /identity/v3/users/c5516a90c978481593de115b69a42a2b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:11.815926 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e644a8b-87bf-484c-abd8-433081317255 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:11.816720 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e644a8b-87bf-484c-abd8-433081317255 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:11.816720 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e644a8b-87bf-484c-abd8-433081317255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:11.836828 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-4e644a8b-87bf-484c-abd8-433081317255 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:11.838156 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-4e644a8b-87bf-484c-abd8-433081317255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:11.926248 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1341/2685] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:30:11.937633 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50d06e-d1bb-4157-8745-3d750123de1b admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:11.939342 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-de50d06e-d1bb-4157-8745-3d750123de1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:12.005637 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50d06e-d1bb-4157-8745-3d750123de1b admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:12.011662 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e8459-e5e3-4c66-9767-a5b24f3d1d08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:12.012520 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e8459-e5e3-4c66-9767-a5b24f3d1d08 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:12.012801 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e8459-e5e3-4c66-9767-a5b24f3d1d08 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:12.013182 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e8459-e5e3-4c66-9767-a5b24f3d1d08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:12.025817 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e8459-e5e3-4c66-9767-a5b24f3d1d08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:12.027401 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e8459-e5e3-4c66-9767-a5b24f3d1d08 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:12.037381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1345/2686] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:12.322181 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e644a8b-87bf-484c-abd8-433081317255 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:12.323743 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-4e644a8b-87bf-484c-abd8-433081317255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:12.336124 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e644a8b-87bf-484c-abd8-433081317255 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:12.341305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ea9bc-7b88-405c-8a13-85927f1a55d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:12.342205 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ea9bc-7b88-405c-8a13-85927f1a55d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:12.342493 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ea9bc-7b88-405c-8a13-85927f1a55d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:12.342773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ea9bc-7b88-405c-8a13-85927f1a55d0 admin admin] PATH_INFO: `/v3/projects/a2891c3f9d95446b951eb7abe7624f6d` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:12.344052 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ea9bc-7b88-405c-8a13-85927f1a55d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2891c3f9d95446b951eb7abe7624f6d)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:12.345978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ea9bc-7b88-405c-8a13-85927f1a55d0 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:12.388259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:12.395076 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-804ea9bc-7b88-405c-8a13-85927f1a55d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2891c3f9d95446b951eb7abe7624f6d', 'request_id': 'req-804ea9bc-7b88-405c-8a13-85927f1a55d0'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:12.401305 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-804ea9bc-7b88-405c-8a13-85927f1a55d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2891c3f9d95446b951eb7abe7624f6d', 'request_id': 'req-804ea9bc-7b88-405c-8a13-85927f1a55d0'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:12.407402 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-804ea9bc-7b88-405c-8a13-85927f1a55d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2891c3f9d95446b951eb7abe7624f6d', 'request_id': 'req-804ea9bc-7b88-405c-8a13-85927f1a55d0'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:12.421488 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-804ea9bc-7b88-405c-8a13-85927f1a55d0 admin admin] The token cache is being invalidate because project a2891c3f9d95446b951eb7abe7624f6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:30:12.421776 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-804ea9bc-7b88-405c-8a13-85927f1a55d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:12.422430 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-804ea9bc-7b88-405c-8a13-85927f1a55d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:12.424046 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1342/2687] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:30:12 2022] DELETE /identity/v3/projects/a2891c3f9d95446b951eb7abe7624f6d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:12.429293 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98239d5-c2df-4744-9ae7-ae66ed7ebaf6 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:12.429581 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98239d5-c2df-4744-9ae7-ae66ed7ebaf6 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:12.429858 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98239d5-c2df-4744-9ae7-ae66ed7ebaf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:12.448776 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-b98239d5-c2df-4744-9ae7-ae66ed7ebaf6 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:12.449906 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-b98239d5-c2df-4744-9ae7-ae66ed7ebaf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:12.514719 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1346/2688] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:30:12.524748 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a671d6d-bb61-46cd-a04b-558143a56cf6 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:12.525121 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a671d6d-bb61-46cd-a04b-558143a56cf6 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:12.525435 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a671d6d-bb61-46cd-a04b-558143a56cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:12.550429 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-1a671d6d-bb61-46cd-a04b-558143a56cf6 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:12.551842 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-1a671d6d-bb61-46cd-a04b-558143a56cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:12.562087 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1343/2689] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:12.568681 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98239d5-c2df-4744-9ae7-ae66ed7ebaf6 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:12.580760 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98239d5-c2df-4744-9ae7-ae66ed7ebaf6 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:12.585648 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d62763-6656-429f-b601-7e9b556b8555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:12.586355 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d62763-6656-429f-b601-7e9b556b8555 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:12.586645 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d62763-6656-429f-b601-7e9b556b8555 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:12.586924 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d62763-6656-429f-b601-7e9b556b8555 admin admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:12.587700 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d62763-6656-429f-b601-7e9b556b8555 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:30:12.588113 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d62763-6656-429f-b601-7e9b556b8555 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:12.589769 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d62763-6656-429f-b601-7e9b556b8555 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:12.602450 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1347/2690] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 15:30:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:30:12.607375 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a671d6d-bb61-46cd-a04b-558143a56cf6 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:12.617272 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a671d6d-bb61-46cd-a04b-558143a56cf6 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:12.623386 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d746ead-828f-4210-a30a-5cb5402ebb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:12.624065 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d746ead-828f-4210-a30a-5cb5402ebb6d admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:12.624373 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d746ead-828f-4210-a30a-5cb5402ebb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:12.624642 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d746ead-828f-4210-a30a-5cb5402ebb6d admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:12.625338 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d746ead-828f-4210-a30a-5cb5402ebb6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:12.627267 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d746ead-828f-4210-a30a-5cb5402ebb6d admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:12.655984 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1344/2691] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:30:12 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:30:12.660583 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d62763-6656-429f-b601-7e9b556b8555 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:12.671636 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d62763-6656-429f-b601-7e9b556b8555 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:12.730050 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f02b82-b730-4096-bef1-c1d12bf43801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:12.730626 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f02b82-b730-4096-bef1-c1d12bf43801 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:12.730905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f02b82-b730-4096-bef1-c1d12bf43801 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:12.731210 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f02b82-b730-4096-bef1-c1d12bf43801 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:12.731921 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f02b82-b730-4096-bef1-c1d12bf43801 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:12.734139 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f02b82-b730-4096-bef1-c1d12bf43801 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:12.757130 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1348/2692] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:30:12 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:30:12.761546 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d746ead-828f-4210-a30a-5cb5402ebb6d admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:12.772681 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d746ead-828f-4210-a30a-5cb5402ebb6d admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:12.778202 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27f01d-1b5b-457a-8f34-3add2f841fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:12.778888 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27f01d-1b5b-457a-8f34-3add2f841fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:12.779220 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27f01d-1b5b-457a-8f34-3add2f841fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:12.779522 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27f01d-1b5b-457a-8f34-3add2f841fc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:12.780263 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a27f01d-1b5b-457a-8f34-3add2f841fc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:12.781920 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a27f01d-1b5b-457a-8f34-3add2f841fc7 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:12.793386 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1345/2693] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:30:12.797743 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f02b82-b730-4096-bef1-c1d12bf43801 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:12.808608 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f02b82-b730-4096-bef1-c1d12bf43801 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:12.813874 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e3337-ca4b-46b2-a421-c2e3ed08b2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:12.814561 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e3337-ca4b-46b2-a421-c2e3ed08b2d4 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:12.814844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e3337-ca4b-46b2-a421-c2e3ed08b2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:12.815177 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e3337-ca4b-46b2-a421-c2e3ed08b2d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:12.815911 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e3337-ca4b-46b2-a421-c2e3ed08b2d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:12.817651 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e3337-ca4b-46b2-a421-c2e3ed08b2d4 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:12.819186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:12.830896 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1349/2694] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:30:12.835085 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27f01d-1b5b-457a-8f34-3add2f841fc7 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:12.845480 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27f01d-1b5b-457a-8f34-3add2f841fc7 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:12.850418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf638f-04a7-4bae-bf6a-8026eebd8980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:12.851391 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf638f-04a7-4bae-bf6a-8026eebd8980 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:12.851674 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf638f-04a7-4bae-bf6a-8026eebd8980 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:12.851955 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf638f-04a7-4bae-bf6a-8026eebd8980 admin admin] PATH_INFO: `/v3/projects/5c01560eceba449ca6c19760b396b439/users/a8b999c73ccf49c19786d3ebbf5a1b20/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:12.868259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cf638f-04a7-4bae-bf6a-8026eebd8980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c01560eceba449ca6c19760b396b439, user_id=a8b999c73ccf49c19786d3ebbf5a1b20, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:12.870918 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cf638f-04a7-4bae-bf6a-8026eebd8980 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:12.885999 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1346/2695] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:30:12 2022] PUT /identity/v3/projects/5c01560eceba449ca6c19760b396b439/users/a8b999c73ccf49c19786d3ebbf5a1b20/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:12.896507 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba767d5f-3c86-49cb-84f6-276ab8b86ed5 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:12.897157 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba767d5f-3c86-49cb-84f6-276ab8b86ed5 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:12.897424 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba767d5f-3c86-49cb-84f6-276ab8b86ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:12.926152 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-ba767d5f-3c86-49cb-84f6-276ab8b86ed5 None None] MFA Rules not processed for user `a8b999c73ccf49c19786d3ebbf5a1b20`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:12.927471 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-ba767d5f-3c86-49cb-84f6-276ab8b86ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:12.993027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1350/2696] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:12 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:12.997625 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf638f-04a7-4bae-bf6a-8026eebd8980 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:13.061446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf638f-04a7-4bae-bf6a-8026eebd8980 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:13.066213 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6d58c-8264-448c-ab6a-77aa62b826c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:13.066916 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6d58c-8264-448c-ab6a-77aa62b826c5 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:13.067245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6d58c-8264-448c-ab6a-77aa62b826c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:13.067520 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6d58c-8264-448c-ab6a-77aa62b826c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:13.068214 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb6d58c-8264-448c-ab6a-77aa62b826c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:13.070131 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb6d58c-8264-448c-ab6a-77aa62b826c5 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:13.098457 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1347/2697] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:30:12 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:30:13.103444 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba767d5f-3c86-49cb-84f6-276ab8b86ed5 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:13.115523 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba767d5f-3c86-49cb-84f6-276ab8b86ed5 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:13.177759 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888b1f5-c3ab-4953-9530-e314149e4dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:13.178446 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888b1f5-c3ab-4953-9530-e314149e4dc4 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:13.178732 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888b1f5-c3ab-4953-9530-e314149e4dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:13.179011 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888b1f5-c3ab-4953-9530-e314149e4dc4 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:13.179766 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2888b1f5-c3ab-4953-9530-e314149e4dc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:13.181733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2888b1f5-c3ab-4953-9530-e314149e4dc4 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:13.202747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1351/2698] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:30:13 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:30:13.207039 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6d58c-8264-448c-ab6a-77aa62b826c5 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:13.218543 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6d58c-8264-448c-ab6a-77aa62b826c5 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:13.224557 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47aa7f1-e3a7-47d2-9156-e1ce9a0d380b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:13.225243 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47aa7f1-e3a7-47d2-9156-e1ce9a0d380b admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:13.225528 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47aa7f1-e3a7-47d2-9156-e1ce9a0d380b admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:13.225804 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47aa7f1-e3a7-47d2-9156-e1ce9a0d380b admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:13.226575 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47aa7f1-e3a7-47d2-9156-e1ce9a0d380b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:13.228541 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47aa7f1-e3a7-47d2-9156-e1ce9a0d380b admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:13.242997 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1348/2699] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:30:13.248179 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888b1f5-c3ab-4953-9530-e314149e4dc4 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:13.262946 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888b1f5-c3ab-4953-9530-e314149e4dc4 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:13.270103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e0c73-6e4d-4d94-bb20-021b7e38c103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:13.271453 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e0c73-6e4d-4d94-bb20-021b7e38c103 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:13.271805 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e0c73-6e4d-4d94-bb20-021b7e38c103 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:13.272220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e0c73-6e4d-4d94-bb20-021b7e38c103 admin admin] PATH_INFO: `/v3/projects/44f6707f33584d7fbdca2b2fa379cf6c/users/8d383cc0fff446e49c7fcc37d6332190/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:13.290178 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e0c73-6e4d-4d94-bb20-021b7e38c103 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44f6707f33584d7fbdca2b2fa379cf6c, user_id=8d383cc0fff446e49c7fcc37d6332190, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:13.292892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e0c73-6e4d-4d94-bb20-021b7e38c103 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:13.307558 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1352/2700] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:30:13 2022] PUT /identity/v3/projects/44f6707f33584d7fbdca2b2fa379cf6c/users/8d383cc0fff446e49c7fcc37d6332190/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:13.312099 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47aa7f1-e3a7-47d2-9156-e1ce9a0d380b admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:13.327803 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47aa7f1-e3a7-47d2-9156-e1ce9a0d380b admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:13.381018 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:13.402482 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346961a-c887-420a-98af-f13d6b9d1cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:13.403234 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346961a-c887-420a-98af-f13d6b9d1cae admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:13.403515 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346961a-c887-420a-98af-f13d6b9d1cae admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:13.403796 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346961a-c887-420a-98af-f13d6b9d1cae admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:13.404525 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5346961a-c887-420a-98af-f13d6b9d1cae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:13.406129 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5346961a-c887-420a-98af-f13d6b9d1cae admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:13.422838 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1349/2701] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:30:13.428248 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e0c73-6e4d-4d94-bb20-021b7e38c103 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:13.438964 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e0c73-6e4d-4d94-bb20-021b7e38c103 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:13.443939 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90547f20-557c-415a-a6e6-3005a4cd602c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:13.444843 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90547f20-557c-415a-a6e6-3005a4cd602c admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:13.445123 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90547f20-557c-415a-a6e6-3005a4cd602c admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:13.445398 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90547f20-557c-415a-a6e6-3005a4cd602c admin admin] PATH_INFO: `/v3/projects/44f6707f33584d7fbdca2b2fa379cf6c/users/8d383cc0fff446e49c7fcc37d6332190/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:13.447546 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90547f20-557c-415a-a6e6-3005a4cd602c admin admin] RBAC: Authorizing `identity:create_grant(project_id=44f6707f33584d7fbdca2b2fa379cf6c, user_id=8d383cc0fff446e49c7fcc37d6332190, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:13.450371 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90547f20-557c-415a-a6e6-3005a4cd602c admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:13.461739 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1353/2702] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:30:13 2022] PUT /identity/v3/projects/44f6707f33584d7fbdca2b2fa379cf6c/users/8d383cc0fff446e49c7fcc37d6332190/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:13.473894 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5538ad-8e82-4fec-a585-f09d2072597b None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:13.474210 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5538ad-8e82-4fec-a585-f09d2072597b None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:13.474487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5538ad-8e82-4fec-a585-f09d2072597b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:13.504173 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-1b5538ad-8e82-4fec-a585-f09d2072597b None None] MFA Rules not processed for user `8d383cc0fff446e49c7fcc37d6332190`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:13.505303 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-1b5538ad-8e82-4fec-a585-f09d2072597b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:13.577901 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1350/2703] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:13 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:13.589829 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90547f20-557c-415a-a6e6-3005a4cd602c admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:13.595186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-90547f20-557c-415a-a6e6-3005a4cd602c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:13.642159 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:13.661726 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90547f20-557c-415a-a6e6-3005a4cd602c admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:13.666135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c7628-89ce-49d7-8945-ff26e529d3ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:13.666811 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c7628-89ce-49d7-8945-ff26e529d3ea service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:13.667123 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c7628-89ce-49d7-8945-ff26e529d3ea service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:13.667410 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c7628-89ce-49d7-8945-ff26e529d3ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:13.677694 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c7628-89ce-49d7-8945-ff26e529d3ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:13.679181 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c7628-89ce-49d7-8945-ff26e529d3ea service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:13.689560 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1354/2704] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:13 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:13.765629 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5538ad-8e82-4fec-a585-f09d2072597b None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:13.776659 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5538ad-8e82-4fec-a585-f09d2072597b None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:13.781167 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd3ec1-3397-4614-aac3-bb3c018fa2aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:13.781878 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd3ec1-3397-4614-aac3-bb3c018fa2aa service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:13.782163 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd3ec1-3397-4614-aac3-bb3c018fa2aa service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:13.782487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd3ec1-3397-4614-aac3-bb3c018fa2aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:13.844976 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd3ec1-3397-4614-aac3-bb3c018fa2aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:13.846455 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd3ec1-3397-4614-aac3-bb3c018fa2aa service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:13.857262 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1351/2705] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:13 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:16.749593 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c7628-89ce-49d7-8945-ff26e529d3ea service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:16.752244 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-ea1c7628-89ce-49d7-8945-ff26e529d3ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:16.834825 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c7628-89ce-49d7-8945-ff26e529d3ea service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:16.841369 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b613158-38b0-403e-9a54-eae0247bc786 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d4a107ed04348b59d20d1703dbe04eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:16.842201 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b613158-38b0-403e-9a54-eae0247bc786 service nova] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:16.842565 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b613158-38b0-403e-9a54-eae0247bc786 service nova] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:16.842912 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b613158-38b0-403e-9a54-eae0247bc786 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:16.844861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-9b613158-38b0-403e-9a54-eae0247bc786 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:16.922374 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b613158-38b0-403e-9a54-eae0247bc786 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:16.924027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b613158-38b0-403e-9a54-eae0247bc786 service nova] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:16.934625 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1355/2706] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:23.852250 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=73 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:26.408683 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd3ec1-3397-4614-aac3-bb3c018fa2aa service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:26.472407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd3ec1-3397-4614-aac3-bb3c018fa2aa service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:26.477599 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4a21f-8e81-450a-a265-5e6dbf1795bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:26.478847 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4a21f-8e81-450a-a265-5e6dbf1795bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:26.479171 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4a21f-8e81-450a-a265-5e6dbf1795bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:26.479424 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4a21f-8e81-450a-a265-5e6dbf1795bb admin admin] PATH_INFO: `/v3/users/a8b999c73ccf49c19786d3ebbf5a1b20` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:26.480744 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4a21f-8e81-450a-a265-5e6dbf1795bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8b999c73ccf49c19786d3ebbf5a1b20)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:26.482689 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4a21f-8e81-450a-a265-5e6dbf1795bb admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:26.535506 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-a1b4a21f-8e81-450a-a265-5e6dbf1795bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8b999c73ccf49c19786d3ebbf5a1b20'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:26.548913 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-a1b4a21f-8e81-450a-a265-5e6dbf1795bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8b999c73ccf49c19786d3ebbf5a1b20'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:26.549663 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-a1b4a21f-8e81-450a-a265-5e6dbf1795bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8b999c73ccf49c19786d3ebbf5a1b20'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:26.550333 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-a1b4a21f-8e81-450a-a265-5e6dbf1795bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8b999c73ccf49c19786d3ebbf5a1b20'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:26.557784 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-a1b4a21f-8e81-450a-a265-5e6dbf1795bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8b999c73ccf49c19786d3ebbf5a1b20'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:26.572454 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1352/2707] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:30:26 2022] DELETE /identity/v3/users/a8b999c73ccf49c19786d3ebbf5a1b20 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:26.577698 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d3a86-30e7-45f6-b57c-67917059bd9c None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:26.578428 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d3a86-30e7-45f6-b57c-67917059bd9c None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:26.578428 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d3a86-30e7-45f6-b57c-67917059bd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:26.595898 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-0a8d3a86-30e7-45f6-b57c-67917059bd9c None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:26.597100 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-0a8d3a86-30e7-45f6-b57c-67917059bd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:26.660689 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1356/2708] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:26.672171 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4a21f-8e81-450a-a265-5e6dbf1795bb admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:26.673718 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-a1b4a21f-8e81-450a-a265-5e6dbf1795bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:26.732900 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4a21f-8e81-450a-a265-5e6dbf1795bb admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:26.739638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50638d5d-fc3e-45d2-a72a-1513d5251d43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:26.740304 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50638d5d-fc3e-45d2-a72a-1513d5251d43 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:26.740592 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50638d5d-fc3e-45d2-a72a-1513d5251d43 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:26.740876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50638d5d-fc3e-45d2-a72a-1513d5251d43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:26.750928 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50638d5d-fc3e-45d2-a72a-1513d5251d43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:26.752410 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50638d5d-fc3e-45d2-a72a-1513d5251d43 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:26.762665 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1353/2709] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:27.058968 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d3a86-30e7-45f6-b57c-67917059bd9c None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:27.060520 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-0a8d3a86-30e7-45f6-b57c-67917059bd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:27.074047 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d3a86-30e7-45f6-b57c-67917059bd9c None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:27.079121 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3fbce4-92a8-4df2-a69c-ba4729c83e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:27.080058 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fbce4-92a8-4df2-a69c-ba4729c83e1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:27.080346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fbce4-92a8-4df2-a69c-ba4729c83e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:27.080627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fbce4-92a8-4df2-a69c-ba4729c83e1d admin admin] PATH_INFO: `/v3/projects/5c01560eceba449ca6c19760b396b439` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:27.081903 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3fbce4-92a8-4df2-a69c-ba4729c83e1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c01560eceba449ca6c19760b396b439)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:27.083864 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3fbce4-92a8-4df2-a69c-ba4729c83e1d admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:27.085580 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:27.141202 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-5f3fbce4-92a8-4df2-a69c-ba4729c83e1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c01560eceba449ca6c19760b396b439', 'request_id': 'req-5f3fbce4-92a8-4df2-a69c-ba4729c83e1d'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:27.147033 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-5f3fbce4-92a8-4df2-a69c-ba4729c83e1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c01560eceba449ca6c19760b396b439', 'request_id': 'req-5f3fbce4-92a8-4df2-a69c-ba4729c83e1d'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:27.156617 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-5f3fbce4-92a8-4df2-a69c-ba4729c83e1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c01560eceba449ca6c19760b396b439', 'request_id': 'req-5f3fbce4-92a8-4df2-a69c-ba4729c83e1d'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:27.168327 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-5f3fbce4-92a8-4df2-a69c-ba4729c83e1d admin admin] The token cache is being invalidate because project 5c01560eceba449ca6c19760b396b439 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:30:27.168620 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-5f3fbce4-92a8-4df2-a69c-ba4729c83e1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:27.169263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-5f3fbce4-92a8-4df2-a69c-ba4729c83e1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:27.170829 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1357/2710] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:30:27 2022] DELETE /identity/v3/projects/5c01560eceba449ca6c19760b396b439 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:27.174829 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50638d5d-fc3e-45d2-a72a-1513d5251d43 service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:27.184534 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50638d5d-fc3e-45d2-a72a-1513d5251d43 service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:27.202639 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:27.248900 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a57846-e628-45e2-8a32-01f28b9ad1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:27.249792 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a57846-e628-45e2-8a32-01f28b9ad1b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:27.250052 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a57846-e628-45e2-8a32-01f28b9ad1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:27.250358 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a57846-e628-45e2-8a32-01f28b9ad1b2 admin admin] PATH_INFO: `/v3/users/8d383cc0fff446e49c7fcc37d6332190` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:27.251653 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a57846-e628-45e2-8a32-01f28b9ad1b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d383cc0fff446e49c7fcc37d6332190)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:27.253593 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a57846-e628-45e2-8a32-01f28b9ad1b2 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:27.302940 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-34a57846-e628-45e2-8a32-01f28b9ad1b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d383cc0fff446e49c7fcc37d6332190'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:27.314901 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-34a57846-e628-45e2-8a32-01f28b9ad1b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d383cc0fff446e49c7fcc37d6332190'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:27.315608 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-34a57846-e628-45e2-8a32-01f28b9ad1b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d383cc0fff446e49c7fcc37d6332190'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:27.316215 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-34a57846-e628-45e2-8a32-01f28b9ad1b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d383cc0fff446e49c7fcc37d6332190'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:27.322962 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-34a57846-e628-45e2-8a32-01f28b9ad1b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d383cc0fff446e49c7fcc37d6332190'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:27.337375 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1354/2711] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:30:27 2022] DELETE /identity/v3/users/8d383cc0fff446e49c7fcc37d6332190 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:27.612003 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3fbce4-92a8-4df2-a69c-ba4729c83e1d admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:27.613845 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-5f3fbce4-92a8-4df2-a69c-ba4729c83e1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:27.632577 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3fbce4-92a8-4df2-a69c-ba4729c83e1d admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:27.694051 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc044d76-34cf-4c41-b7e1-164ec06f2bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:27.694845 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc044d76-34cf-4c41-b7e1-164ec06f2bb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:27.695163 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc044d76-34cf-4c41-b7e1-164ec06f2bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:27.695417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc044d76-34cf-4c41-b7e1-164ec06f2bb1 admin admin] PATH_INFO: `/v3/projects/44f6707f33584d7fbdca2b2fa379cf6c` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:27.696645 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc044d76-34cf-4c41-b7e1-164ec06f2bb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44f6707f33584d7fbdca2b2fa379cf6c)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:27.698605 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc044d76-34cf-4c41-b7e1-164ec06f2bb1 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:27.748482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fc044d76-34cf-4c41-b7e1-164ec06f2bb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44f6707f33584d7fbdca2b2fa379cf6c', 'request_id': 'req-fc044d76-34cf-4c41-b7e1-164ec06f2bb1'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:27.756082 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fc044d76-34cf-4c41-b7e1-164ec06f2bb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44f6707f33584d7fbdca2b2fa379cf6c', 'request_id': 'req-fc044d76-34cf-4c41-b7e1-164ec06f2bb1'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:27.767384 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fc044d76-34cf-4c41-b7e1-164ec06f2bb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44f6707f33584d7fbdca2b2fa379cf6c', 'request_id': 'req-fc044d76-34cf-4c41-b7e1-164ec06f2bb1'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:27.782636 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fc044d76-34cf-4c41-b7e1-164ec06f2bb1 admin admin] The token cache is being invalidate because project 44f6707f33584d7fbdca2b2fa379cf6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:30:27.782905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fc044d76-34cf-4c41-b7e1-164ec06f2bb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:27.783710 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fc044d76-34cf-4c41-b7e1-164ec06f2bb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:27.785391 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1358/2712] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:30:27 2022] DELETE /identity/v3/projects/44f6707f33584d7fbdca2b2fa379cf6c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:27.791754 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eee748-4f30-4cd3-9572-abf1ff01af91 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:27.792253 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eee748-4f30-4cd3-9572-abf1ff01af91 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:27.792363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eee748-4f30-4cd3-9572-abf1ff01af91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:27.817269 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-66eee748-4f30-4cd3-9572-abf1ff01af91 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:27.818494 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-66eee748-4f30-4cd3-9572-abf1ff01af91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:27.888964 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1355/2713] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:27.896945 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae6250-8cf8-45af-bc3a-6362e87743f2 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:27.897235 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae6250-8cf8-45af-bc3a-6362e87743f2 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:27.897516 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae6250-8cf8-45af-bc3a-6362e87743f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:27.916508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-d8ae6250-8cf8-45af-bc3a-6362e87743f2 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:27.917718 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-d8ae6250-8cf8-45af-bc3a-6362e87743f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:27.926177 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1359/2714] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:27.930461 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eee748-4f30-4cd3-9572-abf1ff01af91 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:27.940217 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eee748-4f30-4cd3-9572-abf1ff01af91 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:27.945068 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8946230-cf02-41ca-9eed-df735d47cc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:27.945668 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8946230-cf02-41ca-9eed-df735d47cc8a admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:27.945953 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8946230-cf02-41ca-9eed-df735d47cc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:27.946227 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8946230-cf02-41ca-9eed-df735d47cc8a admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:27.947013 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8946230-cf02-41ca-9eed-df735d47cc8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:30:27.947355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8946230-cf02-41ca-9eed-df735d47cc8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:27.948967 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8946230-cf02-41ca-9eed-df735d47cc8a admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:27.960743 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1356/2715] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 15:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:30:27.967125 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae6250-8cf8-45af-bc3a-6362e87743f2 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:27.980307 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae6250-8cf8-45af-bc3a-6362e87743f2 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:27.985749 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43434183-a6bf-43c9-928d-b47b59ecd29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:27.986426 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43434183-a6bf-43c9-928d-b47b59ecd29b admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:27.986714 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43434183-a6bf-43c9-928d-b47b59ecd29b admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:27.986991 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43434183-a6bf-43c9-928d-b47b59ecd29b admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:27.987728 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43434183-a6bf-43c9-928d-b47b59ecd29b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:27.989678 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43434183-a6bf-43c9-928d-b47b59ecd29b admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:28.000250 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:28.019787 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1360/2716] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:30:27 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:30:28.024018 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8946230-cf02-41ca-9eed-df735d47cc8a admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:28.035591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8946230-cf02-41ca-9eed-df735d47cc8a admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:28.064092 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:28.098389 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb97fd6-ecff-4bac-879c-03853ead90ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:28.099125 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb97fd6-ecff-4bac-879c-03853ead90ec admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:28.100322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb97fd6-ecff-4bac-879c-03853ead90ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:28.100322 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb97fd6-ecff-4bac-879c-03853ead90ec admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:28.100593 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb97fd6-ecff-4bac-879c-03853ead90ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:28.102306 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb97fd6-ecff-4bac-879c-03853ead90ec admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:28.124916 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1357/2717] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:30:28 2022] POST /identity/v3/users => generated 283 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:30:28.129942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43434183-a6bf-43c9-928d-b47b59ecd29b admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:28.142155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43434183-a6bf-43c9-928d-b47b59ecd29b admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:28.148140 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e8318-584d-4d11-8de6-922a7f5e38d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:28.149144 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e8318-584d-4d11-8de6-922a7f5e38d3 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:28.149540 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e8318-584d-4d11-8de6-922a7f5e38d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:28.149935 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e8318-584d-4d11-8de6-922a7f5e38d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:28.151164 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88e8318-584d-4d11-8de6-922a7f5e38d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:28.153705 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88e8318-584d-4d11-8de6-922a7f5e38d3 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:28.167880 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1361/2718] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:30:28.172448 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb97fd6-ecff-4bac-879c-03853ead90ec admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:28.183352 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb97fd6-ecff-4bac-879c-03853ead90ec admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:28.188547 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed23e445-0e3c-42ff-b091-5b8713736e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:28.189232 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23e445-0e3c-42ff-b091-5b8713736e4a admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:28.189516 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23e445-0e3c-42ff-b091-5b8713736e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:28.189794 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23e445-0e3c-42ff-b091-5b8713736e4a admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:28.190529 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed23e445-0e3c-42ff-b091-5b8713736e4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:28.192172 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed23e445-0e3c-42ff-b091-5b8713736e4a admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:28.202702 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1358/2719] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:30:28.207021 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e8318-584d-4d11-8de6-922a7f5e38d3 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:28.218435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e8318-584d-4d11-8de6-922a7f5e38d3 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:28.223397 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef6b436-018a-41b2-a7f4-f75b7ffdbe24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:28.224297 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6b436-018a-41b2-a7f4-f75b7ffdbe24 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:28.224579 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6b436-018a-41b2-a7f4-f75b7ffdbe24 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:28.224851 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6b436-018a-41b2-a7f4-f75b7ffdbe24 admin admin] PATH_INFO: `/v3/projects/50defaa66c6b4dd8be85f079ad012022/users/4d8caa68a9e541fbbec1927487ef7118/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:28.240127 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef6b436-018a-41b2-a7f4-f75b7ffdbe24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50defaa66c6b4dd8be85f079ad012022, user_id=4d8caa68a9e541fbbec1927487ef7118, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:28.242767 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef6b436-018a-41b2-a7f4-f75b7ffdbe24 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:28.256250 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1362/2720] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:30:28 2022] PUT /identity/v3/projects/50defaa66c6b4dd8be85f079ad012022/users/4d8caa68a9e541fbbec1927487ef7118/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:28.267356 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfaac88-b17c-43dc-8177-2969aa61ef16 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:28.267653 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfaac88-b17c-43dc-8177-2969aa61ef16 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:28.267937 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfaac88-b17c-43dc-8177-2969aa61ef16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:28.295934 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-5bfaac88-b17c-43dc-8177-2969aa61ef16 None None] MFA Rules not processed for user `4d8caa68a9e541fbbec1927487ef7118`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:28.297186 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-5bfaac88-b17c-43dc-8177-2969aa61ef16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:28.375585 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1359/2721] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:28 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:28.382148 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef6b436-018a-41b2-a7f4-f75b7ffdbe24 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:28.466344 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef6b436-018a-41b2-a7f4-f75b7ffdbe24 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:28.471980 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a56581c-8ef4-47f5-b0e3-ab82c6ba82c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:28.472685 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56581c-8ef4-47f5-b0e3-ab82c6ba82c5 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:28.472948 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56581c-8ef4-47f5-b0e3-ab82c6ba82c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:28.473233 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56581c-8ef4-47f5-b0e3-ab82c6ba82c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:28.473925 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a56581c-8ef4-47f5-b0e3-ab82c6ba82c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:28.475926 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a56581c-8ef4-47f5-b0e3-ab82c6ba82c5 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:28.511012 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1363/2722] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:30:28 2022] POST /identity/v3/projects => generated 365 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:30:28.512387 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfaac88-b17c-43dc-8177-2969aa61ef16 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:28.524342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfaac88-b17c-43dc-8177-2969aa61ef16 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:28.585960 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3479fcb-01a0-4cf5-a019-ee8b81c01841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:28.586650 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3479fcb-01a0-4cf5-a019-ee8b81c01841 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:28.586936 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3479fcb-01a0-4cf5-a019-ee8b81c01841 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:28.587245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3479fcb-01a0-4cf5-a019-ee8b81c01841 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:28.588110 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3479fcb-01a0-4cf5-a019-ee8b81c01841 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:28.590086 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3479fcb-01a0-4cf5-a019-ee8b81c01841 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:28.614999 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1360/2723] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:30:28 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:30:28.619978 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a56581c-8ef4-47f5-b0e3-ab82c6ba82c5 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:28.635836 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a56581c-8ef4-47f5-b0e3-ab82c6ba82c5 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:28.642021 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0adf5c6-ba98-4fe0-8d2f-2e252e4478e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:28.642701 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0adf5c6-ba98-4fe0-8d2f-2e252e4478e0 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:28.642984 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0adf5c6-ba98-4fe0-8d2f-2e252e4478e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:28.643289 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0adf5c6-ba98-4fe0-8d2f-2e252e4478e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:28.644027 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0adf5c6-ba98-4fe0-8d2f-2e252e4478e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:28.645642 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0adf5c6-ba98-4fe0-8d2f-2e252e4478e0 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:28.657323 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1364/2724] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:30:28.662861 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3479fcb-01a0-4cf5-a019-ee8b81c01841 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:28.675246 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3479fcb-01a0-4cf5-a019-ee8b81c01841 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:28.680394 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dae3f3-7eb0-49ec-91ba-42a71aaa38ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:28.681299 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dae3f3-7eb0-49ec-91ba-42a71aaa38ec admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:28.681580 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dae3f3-7eb0-49ec-91ba-42a71aaa38ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:28.681855 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dae3f3-7eb0-49ec-91ba-42a71aaa38ec admin admin] PATH_INFO: `/v3/projects/514d7919049c46f58c99b0946f02a264/users/49e0573c1fe9460495c3f37b2bc0a8c5/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:28.704462 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dae3f3-7eb0-49ec-91ba-42a71aaa38ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=514d7919049c46f58c99b0946f02a264, user_id=49e0573c1fe9460495c3f37b2bc0a8c5, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:28.707248 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dae3f3-7eb0-49ec-91ba-42a71aaa38ec admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:28.722952 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1361/2725] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:30:28 2022] PUT /identity/v3/projects/514d7919049c46f58c99b0946f02a264/users/49e0573c1fe9460495c3f37b2bc0a8c5/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:28.727274 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0adf5c6-ba98-4fe0-8d2f-2e252e4478e0 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:28.742566 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0adf5c6-ba98-4fe0-8d2f-2e252e4478e0 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:28.815265 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566f88f-4a18-44be-bfd3-539f98ea092b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:28.815963 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566f88f-4a18-44be-bfd3-539f98ea092b admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:28.816263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566f88f-4a18-44be-bfd3-539f98ea092b admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:28.816541 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566f88f-4a18-44be-bfd3-539f98ea092b admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:28.817293 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9566f88f-4a18-44be-bfd3-539f98ea092b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:28.818941 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9566f88f-4a18-44be-bfd3-539f98ea092b admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:28.829669 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1365/2726] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:30:28.834121 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dae3f3-7eb0-49ec-91ba-42a71aaa38ec admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:28.840371 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:28.847653 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dae3f3-7eb0-49ec-91ba-42a71aaa38ec admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:28.853534 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d6b4f1-cb16-45b5-9009-96b1ba113c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:28.854449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6b4f1-cb16-45b5-9009-96b1ba113c50 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:28.854733 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6b4f1-cb16-45b5-9009-96b1ba113c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:28.855012 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6b4f1-cb16-45b5-9009-96b1ba113c50 admin admin] PATH_INFO: `/v3/projects/514d7919049c46f58c99b0946f02a264/users/49e0573c1fe9460495c3f37b2bc0a8c5/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:28.857032 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d6b4f1-cb16-45b5-9009-96b1ba113c50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=514d7919049c46f58c99b0946f02a264, user_id=49e0573c1fe9460495c3f37b2bc0a8c5, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:28.859739 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d6b4f1-cb16-45b5-9009-96b1ba113c50 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:28.870355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1362/2727] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:30:28 2022] PUT /identity/v3/projects/514d7919049c46f58c99b0946f02a264/users/49e0573c1fe9460495c3f37b2bc0a8c5/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:28.881950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8a946-1876-4afc-bb2e-f7c82577b947 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:28.882146 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8a946-1876-4afc-bb2e-f7c82577b947 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:28.882444 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8a946-1876-4afc-bb2e-f7c82577b947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:28.914498 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-5cc8a946-1876-4afc-bb2e-f7c82577b947 None None] MFA Rules not processed for user `49e0573c1fe9460495c3f37b2bc0a8c5`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:28.916232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-5cc8a946-1876-4afc-bb2e-f7c82577b947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:28.931887 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:28.996740 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1366/2728] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:28 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:30:29.008363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d6b4f1-cb16-45b5-9009-96b1ba113c50 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:29.010230 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-74d6b4f1-cb16-45b5-9009-96b1ba113c50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:29.085327 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d6b4f1-cb16-45b5-9009-96b1ba113c50 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:29.089271 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8166b1c1-9ad9-48fe-8cbf-6501f9508bd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:29.089926 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166b1c1-9ad9-48fe-8cbf-6501f9508bd2 service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:29.090232 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166b1c1-9ad9-48fe-8cbf-6501f9508bd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:29.090515 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166b1c1-9ad9-48fe-8cbf-6501f9508bd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:29.101105 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8166b1c1-9ad9-48fe-8cbf-6501f9508bd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:29.102988 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8166b1c1-9ad9-48fe-8cbf-6501f9508bd2 service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:29.114112 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1363/2729] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:29 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:29.197906 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8a946-1876-4afc-bb2e-f7c82577b947 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:29.210089 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8a946-1876-4afc-bb2e-f7c82577b947 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:29.214594 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf2686-c2bc-401f-b540-fad0779dd4c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:29.215291 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf2686-c2bc-401f-b540-fad0779dd4c3 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:29.215622 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf2686-c2bc-401f-b540-fad0779dd4c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:29.215887 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf2686-c2bc-401f-b540-fad0779dd4c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:29.279668 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf2686-c2bc-401f-b540-fad0779dd4c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:29.281150 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf2686-c2bc-401f-b540-fad0779dd4c3 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:29.291116 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1367/2730] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:29 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:32.058583 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8166b1c1-9ad9-48fe-8cbf-6501f9508bd2 service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:32.132806 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8166b1c1-9ad9-48fe-8cbf-6501f9508bd2 service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:32.139522 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b70fdd-7888-43bf-96e6-96c21986c2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:32.141662 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b70fdd-7888-43bf-96e6-96c21986c2df admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:32.142451 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b70fdd-7888-43bf-96e6-96c21986c2df admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:32.143411 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b70fdd-7888-43bf-96e6-96c21986c2df admin admin] PATH_INFO: `/v3/users/4d8caa68a9e541fbbec1927487ef7118` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:32.146543 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b70fdd-7888-43bf-96e6-96c21986c2df admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d8caa68a9e541fbbec1927487ef7118)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:32.150965 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b70fdd-7888-43bf-96e6-96c21986c2df admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:32.217430 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-29b70fdd-7888-43bf-96e6-96c21986c2df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d8caa68a9e541fbbec1927487ef7118'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:32.230532 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-29b70fdd-7888-43bf-96e6-96c21986c2df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d8caa68a9e541fbbec1927487ef7118'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:32.231448 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-29b70fdd-7888-43bf-96e6-96c21986c2df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d8caa68a9e541fbbec1927487ef7118'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:32.232127 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-29b70fdd-7888-43bf-96e6-96c21986c2df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d8caa68a9e541fbbec1927487ef7118'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:32.244317 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-29b70fdd-7888-43bf-96e6-96c21986c2df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d8caa68a9e541fbbec1927487ef7118'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:32.265615 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1364/2731] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:30:32 2022] DELETE /identity/v3/users/4d8caa68a9e541fbbec1927487ef7118 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:32.271380 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff554311-f549-4f1a-bd30-4910615cab14 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:32.271905 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff554311-f549-4f1a-bd30-4910615cab14 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:32.272024 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff554311-f549-4f1a-bd30-4910615cab14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:32.296700 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-ff554311-f549-4f1a-bd30-4910615cab14 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:32.298091 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-ff554311-f549-4f1a-bd30-4910615cab14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:32.371135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1368/2732] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:32.383135 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b70fdd-7888-43bf-96e6-96c21986c2df admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:32.385190 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-29b70fdd-7888-43bf-96e6-96c21986c2df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:32.403209 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:32.454076 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b70fdd-7888-43bf-96e6-96c21986c2df admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:32.458108 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646527c-fe95-40d5-9e4e-25b5aecc1a16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:32.458785 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646527c-fe95-40d5-9e4e-25b5aecc1a16 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:32.459077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646527c-fe95-40d5-9e4e-25b5aecc1a16 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:32.459377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646527c-fe95-40d5-9e4e-25b5aecc1a16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:32.469493 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7646527c-fe95-40d5-9e4e-25b5aecc1a16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:32.470996 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7646527c-fe95-40d5-9e4e-25b5aecc1a16 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:32.483883 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1365/2733] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:32.858006 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff554311-f549-4f1a-bd30-4910615cab14 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:32.859620 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-ff554311-f549-4f1a-bd30-4910615cab14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:32.875276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff554311-f549-4f1a-bd30-4910615cab14 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:32.882520 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aaf7ca-2d75-4260-bcbf-26f68f480f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:32.884157 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aaf7ca-2d75-4260-bcbf-26f68f480f49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:32.884685 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aaf7ca-2d75-4260-bcbf-26f68f480f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:32.884946 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aaf7ca-2d75-4260-bcbf-26f68f480f49 admin admin] PATH_INFO: `/v3/projects/50defaa66c6b4dd8be85f079ad012022` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:32.886287 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aaf7ca-2d75-4260-bcbf-26f68f480f49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50defaa66c6b4dd8be85f079ad012022)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:32.888675 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aaf7ca-2d75-4260-bcbf-26f68f480f49 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:32.952996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:32.956713 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-27aaf7ca-2d75-4260-bcbf-26f68f480f49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50defaa66c6b4dd8be85f079ad012022', 'request_id': 'req-27aaf7ca-2d75-4260-bcbf-26f68f480f49'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:32.962830 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-27aaf7ca-2d75-4260-bcbf-26f68f480f49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50defaa66c6b4dd8be85f079ad012022', 'request_id': 'req-27aaf7ca-2d75-4260-bcbf-26f68f480f49'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:32.972213 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-27aaf7ca-2d75-4260-bcbf-26f68f480f49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50defaa66c6b4dd8be85f079ad012022', 'request_id': 'req-27aaf7ca-2d75-4260-bcbf-26f68f480f49'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:32.989493 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-27aaf7ca-2d75-4260-bcbf-26f68f480f49 admin admin] The token cache is being invalidate because project 50defaa66c6b4dd8be85f079ad012022 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:30:32.989916 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-27aaf7ca-2d75-4260-bcbf-26f68f480f49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:32.990711 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-27aaf7ca-2d75-4260-bcbf-26f68f480f49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:32.992633 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1369/2734] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:30:32 2022] DELETE /identity/v3/projects/50defaa66c6b4dd8be85f079ad012022 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:30:32.997118 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646527c-fe95-40d5-9e4e-25b5aecc1a16 service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:33.017501 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646527c-fe95-40d5-9e4e-25b5aecc1a16 service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:33.097100 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea3e75-73d2-4caf-bd55-40cdf4713bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:33.098262 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea3e75-73d2-4caf-bd55-40cdf4713bec admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:33.098548 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea3e75-73d2-4caf-bd55-40cdf4713bec admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:33.098829 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea3e75-73d2-4caf-bd55-40cdf4713bec admin admin] PATH_INFO: `/v3/users/49e0573c1fe9460495c3f37b2bc0a8c5` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:33.100248 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea3e75-73d2-4caf-bd55-40cdf4713bec admin admin] RBAC: Authorizing `identity:delete_user(user_id=49e0573c1fe9460495c3f37b2bc0a8c5)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:33.102201 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea3e75-73d2-4caf-bd55-40cdf4713bec admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:33.165381 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-95ea3e75-73d2-4caf-bd55-40cdf4713bec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49e0573c1fe9460495c3f37b2bc0a8c5'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:33.177495 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-95ea3e75-73d2-4caf-bd55-40cdf4713bec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49e0573c1fe9460495c3f37b2bc0a8c5'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:33.178255 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-95ea3e75-73d2-4caf-bd55-40cdf4713bec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49e0573c1fe9460495c3f37b2bc0a8c5'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:33.178895 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-95ea3e75-73d2-4caf-bd55-40cdf4713bec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49e0573c1fe9460495c3f37b2bc0a8c5'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:33.187496 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-95ea3e75-73d2-4caf-bd55-40cdf4713bec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49e0573c1fe9460495c3f37b2bc0a8c5'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:33.202723 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1366/2735] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:30:32 2022] DELETE /identity/v3/users/49e0573c1fe9460495c3f37b2bc0a8c5 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:33.516969 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aaf7ca-2d75-4260-bcbf-26f68f480f49 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:33.518523 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-27aaf7ca-2d75-4260-bcbf-26f68f480f49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:33.534590 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aaf7ca-2d75-4260-bcbf-26f68f480f49 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:33.623159 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f66e80a-c1ad-450c-9983-99121902b3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:33.629437 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66e80a-c1ad-450c-9983-99121902b3e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:33.629714 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66e80a-c1ad-450c-9983-99121902b3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:33.629988 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66e80a-c1ad-450c-9983-99121902b3e9 admin admin] PATH_INFO: `/v3/projects/514d7919049c46f58c99b0946f02a264` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:33.631483 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f66e80a-c1ad-450c-9983-99121902b3e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=514d7919049c46f58c99b0946f02a264)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:33.633422 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f66e80a-c1ad-450c-9983-99121902b3e9 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:33.689432 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-8f66e80a-c1ad-450c-9983-99121902b3e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '514d7919049c46f58c99b0946f02a264', 'request_id': 'req-8f66e80a-c1ad-450c-9983-99121902b3e9'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:33.695238 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-8f66e80a-c1ad-450c-9983-99121902b3e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '514d7919049c46f58c99b0946f02a264', 'request_id': 'req-8f66e80a-c1ad-450c-9983-99121902b3e9'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:33.706561 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-8f66e80a-c1ad-450c-9983-99121902b3e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '514d7919049c46f58c99b0946f02a264', 'request_id': 'req-8f66e80a-c1ad-450c-9983-99121902b3e9'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:33.720395 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-8f66e80a-c1ad-450c-9983-99121902b3e9 admin admin] The token cache is being invalidate because project 514d7919049c46f58c99b0946f02a264 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:30:33.720750 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-8f66e80a-c1ad-450c-9983-99121902b3e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:33.722690 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-8f66e80a-c1ad-450c-9983-99121902b3e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:33.724387 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1370/2736] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:30:33 2022] DELETE /identity/v3/projects/514d7919049c46f58c99b0946f02a264 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:33.731300 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4864706-9418-43d8-b20b-42225630fc17 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:33.731756 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4864706-9418-43d8-b20b-42225630fc17 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:33.732082 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4864706-9418-43d8-b20b-42225630fc17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:33.754674 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-d4864706-9418-43d8-b20b-42225630fc17 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:33.756282 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-d4864706-9418-43d8-b20b-42225630fc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:33.797527 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:33.842319 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1367/2737] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:33.852414 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc2b92-a99d-4084-bcf1-9c80d90380cc None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:33.853063 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc2b92-a99d-4084-bcf1-9c80d90380cc None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:33.853382 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc2b92-a99d-4084-bcf1-9c80d90380cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:33.872764 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-a5fc2b92-a99d-4084-bcf1-9c80d90380cc None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:33.874132 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-a5fc2b92-a99d-4084-bcf1-9c80d90380cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:33.882859 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1371/2738] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:33.888206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4864706-9418-43d8-b20b-42225630fc17 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:33.902812 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4864706-9418-43d8-b20b-42225630fc17 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:33.911695 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c5128-b4df-4cf9-8ba5-5d1d7d5c0a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:33.912994 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c5128-b4df-4cf9-8ba5-5d1d7d5c0a34 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:33.913455 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c5128-b4df-4cf9-8ba5-5d1d7d5c0a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:33.913838 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c5128-b4df-4cf9-8ba5-5d1d7d5c0a34 admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:33.915089 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13c5128-b4df-4cf9-8ba5-5d1d7d5c0a34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:30:33.915528 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13c5128-b4df-4cf9-8ba5-5d1d7d5c0a34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:33.917220 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13c5128-b4df-4cf9-8ba5-5d1d7d5c0a34 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:33.931890 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1368/2739] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 15:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:30:33.938000 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc2b92-a99d-4084-bcf1-9c80d90380cc None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:33.952007 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc2b92-a99d-4084-bcf1-9c80d90380cc None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:33.959359 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158845d-12b4-46e7-8412-30d4207534a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:33.960059 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158845d-12b4-46e7-8412-30d4207534a1 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:33.960353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158845d-12b4-46e7-8412-30d4207534a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:33.960677 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158845d-12b4-46e7-8412-30d4207534a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:33.961398 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5158845d-12b4-46e7-8412-30d4207534a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:33.963393 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5158845d-12b4-46e7-8412-30d4207534a1 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:33.999606 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1372/2740] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:30:33 2022] POST /identity/v3/projects => generated 367 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:30:34.004572 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c5128-b4df-4cf9-8ba5-5d1d7d5c0a34 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:34.017827 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c5128-b4df-4cf9-8ba5-5d1d7d5c0a34 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:34.081477 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5762e7-c0ef-4e45-ab5a-30eb14309762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:34.082163 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5762e7-c0ef-4e45-ab5a-30eb14309762 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:34.082449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5762e7-c0ef-4e45-ab5a-30eb14309762 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:34.082732 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5762e7-c0ef-4e45-ab5a-30eb14309762 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:34.083485 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5762e7-c0ef-4e45-ab5a-30eb14309762 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:34.085705 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5762e7-c0ef-4e45-ab5a-30eb14309762 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:34.109145 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1369/2741] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:30:34 2022] POST /identity/v3/users => generated 284 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:30:34.113706 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158845d-12b4-46e7-8412-30d4207534a1 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:34.125712 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158845d-12b4-46e7-8412-30d4207534a1 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:34.130977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa980a2f-20ec-4363-a45d-fcfb9d2034bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:34.131732 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa980a2f-20ec-4363-a45d-fcfb9d2034bd admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:34.132038 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa980a2f-20ec-4363-a45d-fcfb9d2034bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:34.132320 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa980a2f-20ec-4363-a45d-fcfb9d2034bd admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:34.133150 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa980a2f-20ec-4363-a45d-fcfb9d2034bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:34.134963 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa980a2f-20ec-4363-a45d-fcfb9d2034bd admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:34.147496 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1373/2742] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:30:34.152676 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5762e7-c0ef-4e45-ab5a-30eb14309762 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:34.166325 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5762e7-c0ef-4e45-ab5a-30eb14309762 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:34.171479 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca2c89b-498a-4e06-be40-9aa0623b0370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:34.172451 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2c89b-498a-4e06-be40-9aa0623b0370 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:34.172793 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2c89b-498a-4e06-be40-9aa0623b0370 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:34.173119 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2c89b-498a-4e06-be40-9aa0623b0370 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:34.174057 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca2c89b-498a-4e06-be40-9aa0623b0370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:34.175871 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca2c89b-498a-4e06-be40-9aa0623b0370 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:34.187225 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1370/2743] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:30:34.191672 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa980a2f-20ec-4363-a45d-fcfb9d2034bd admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:34.202681 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa980a2f-20ec-4363-a45d-fcfb9d2034bd admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:34.210313 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c81ecb9-73de-4c15-adc3-0f9e75ed7248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:34.211262 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81ecb9-73de-4c15-adc3-0f9e75ed7248 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:34.211546 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81ecb9-73de-4c15-adc3-0f9e75ed7248 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:34.211818 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81ecb9-73de-4c15-adc3-0f9e75ed7248 admin admin] PATH_INFO: `/v3/projects/7034f3ae26504c50bfeb43af8032a477/users/aafe6c356bea4d00a532dcea417f2f3e/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:34.232365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c81ecb9-73de-4c15-adc3-0f9e75ed7248 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7034f3ae26504c50bfeb43af8032a477, user_id=aafe6c356bea4d00a532dcea417f2f3e, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:34.235019 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c81ecb9-73de-4c15-adc3-0f9e75ed7248 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:34.246306 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:34.253464 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1374/2744] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:30:34 2022] PUT /identity/v3/projects/7034f3ae26504c50bfeb43af8032a477/users/aafe6c356bea4d00a532dcea417f2f3e/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:34.269398 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9ff6b-5474-47a3-8da1-ad9f962cac51 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:34.269878 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9ff6b-5474-47a3-8da1-ad9f962cac51 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:34.270243 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9ff6b-5474-47a3-8da1-ad9f962cac51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:34.300056 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-1ef9ff6b-5474-47a3-8da1-ad9f962cac51 None None] MFA Rules not processed for user `aafe6c356bea4d00a532dcea417f2f3e`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:34.301317 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-1ef9ff6b-5474-47a3-8da1-ad9f962cac51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:34.404285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1371/2745] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:34 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:34.405628 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c81ecb9-73de-4c15-adc3-0f9e75ed7248 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:34.621379 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c81ecb9-73de-4c15-adc3-0f9e75ed7248 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:34.630049 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64f82b-f98e-40f8-a83d-92f58729b518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:34.630727 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64f82b-f98e-40f8-a83d-92f58729b518 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:34.631008 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64f82b-f98e-40f8-a83d-92f58729b518 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:34.631472 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64f82b-f98e-40f8-a83d-92f58729b518 admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:34.632663 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a64f82b-f98e-40f8-a83d-92f58729b518 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:34.635601 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a64f82b-f98e-40f8-a83d-92f58729b518 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:34.670915 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1375/2746] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:30:34 2022] POST /identity/v3/projects => generated 367 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:30:34.675663 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef9ff6b-5474-47a3-8da1-ad9f962cac51 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:34.691756 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef9ff6b-5474-47a3-8da1-ad9f962cac51 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:34.731797 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:34.779790 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580f81d-ffd9-454b-8f16-ec48f63e8d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:34.780766 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580f81d-ffd9-454b-8f16-ec48f63e8d39 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:34.781274 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580f81d-ffd9-454b-8f16-ec48f63e8d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:34.781753 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580f81d-ffd9-454b-8f16-ec48f63e8d39 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:34.782965 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7580f81d-ffd9-454b-8f16-ec48f63e8d39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:34.785823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7580f81d-ffd9-454b-8f16-ec48f63e8d39 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:34.817875 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1372/2747] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:30:34 2022] POST /identity/v3/users => generated 290 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:30:34.824420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64f82b-f98e-40f8-a83d-92f58729b518 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:34.843245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64f82b-f98e-40f8-a83d-92f58729b518 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:34.850382 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87d8cd-9de7-45b0-b85d-f861d444cf9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:34.851371 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87d8cd-9de7-45b0-b85d-f861d444cf9e admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:34.851765 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87d8cd-9de7-45b0-b85d-f861d444cf9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:34.852070 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87d8cd-9de7-45b0-b85d-f861d444cf9e admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:34.853200 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87d8cd-9de7-45b0-b85d-f861d444cf9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:34.854907 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87d8cd-9de7-45b0-b85d-f861d444cf9e admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:34.867497 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1376/2748] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:30:34.872503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580f81d-ffd9-454b-8f16-ec48f63e8d39 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:34.883248 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580f81d-ffd9-454b-8f16-ec48f63e8d39 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:34.889374 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc73be5-6835-4ed1-a929-eb258176278d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:34.890471 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc73be5-6835-4ed1-a929-eb258176278d admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:34.890805 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc73be5-6835-4ed1-a929-eb258176278d admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:34.891137 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc73be5-6835-4ed1-a929-eb258176278d admin admin] PATH_INFO: `/v3/projects/b63e4bc464ae43b1a1d0b3dbaaa24edd/users/42d753eb491840fcbd5c728c16ba40ee/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:34.909050 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc73be5-6835-4ed1-a929-eb258176278d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b63e4bc464ae43b1a1d0b3dbaaa24edd, user_id=42d753eb491840fcbd5c728c16ba40ee, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:34.912412 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc73be5-6835-4ed1-a929-eb258176278d admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:34.929848 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1373/2749] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:30:34 2022] PUT /identity/v3/projects/b63e4bc464ae43b1a1d0b3dbaaa24edd/users/42d753eb491840fcbd5c728c16ba40ee/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:30:34.935134 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87d8cd-9de7-45b0-b85d-f861d444cf9e admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:34.950421 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87d8cd-9de7-45b0-b85d-f861d444cf9e admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:35.031572 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb36c8c-24fb-42e3-b11b-be92ab80794f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:35.032167 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb36c8c-24fb-42e3-b11b-be92ab80794f admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:35.032588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb36c8c-24fb-42e3-b11b-be92ab80794f admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:35.032991 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb36c8c-24fb-42e3-b11b-be92ab80794f admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:35.033770 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb36c8c-24fb-42e3-b11b-be92ab80794f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:35.035415 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb36c8c-24fb-42e3-b11b-be92ab80794f admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:35.046263 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1377/2750] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:30:35.051205 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc73be5-6835-4ed1-a929-eb258176278d admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:35.062878 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc73be5-6835-4ed1-a929-eb258176278d admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:35.067843 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d05a8e-6f83-409d-96b5-d8bd6304c21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:35.068750 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d05a8e-6f83-409d-96b5-d8bd6304c21f admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:35.069025 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d05a8e-6f83-409d-96b5-d8bd6304c21f admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:35.069302 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d05a8e-6f83-409d-96b5-d8bd6304c21f admin admin] PATH_INFO: `/v3/projects/b63e4bc464ae43b1a1d0b3dbaaa24edd/users/42d753eb491840fcbd5c728c16ba40ee/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:35.071375 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d05a8e-6f83-409d-96b5-d8bd6304c21f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b63e4bc464ae43b1a1d0b3dbaaa24edd, user_id=42d753eb491840fcbd5c728c16ba40ee, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:35.074110 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d05a8e-6f83-409d-96b5-d8bd6304c21f admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:35.090473 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1374/2751] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:30:35 2022] PUT /identity/v3/projects/b63e4bc464ae43b1a1d0b3dbaaa24edd/users/42d753eb491840fcbd5c728c16ba40ee/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:35.158821 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df371a67-244b-4e3a-a322-37f1bd32340d None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:35.159219 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df371a67-244b-4e3a-a322-37f1bd32340d None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:35.159594 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df371a67-244b-4e3a-a322-37f1bd32340d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:35.190273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-df371a67-244b-4e3a-a322-37f1bd32340d None None] MFA Rules not processed for user `42d753eb491840fcbd5c728c16ba40ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:35.191559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-df371a67-244b-4e3a-a322-37f1bd32340d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:35.260945 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:35.275291 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1378/2752] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:35 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:35.296838 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d05a8e-6f83-409d-96b5-d8bd6304c21f admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:35.299489 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-b9d05a8e-6f83-409d-96b5-d8bd6304c21f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:35.383630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d05a8e-6f83-409d-96b5-d8bd6304c21f admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:35.389834 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6565fe83-6efa-4061-97f3-3dc0fd1f7b57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:35.390674 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565fe83-6efa-4061-97f3-3dc0fd1f7b57 service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:35.390969 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565fe83-6efa-4061-97f3-3dc0fd1f7b57 service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:35.391334 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565fe83-6efa-4061-97f3-3dc0fd1f7b57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:35.401757 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6565fe83-6efa-4061-97f3-3dc0fd1f7b57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:35.403257 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6565fe83-6efa-4061-97f3-3dc0fd1f7b57 service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:35.413299 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1375/2753] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:35 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:30:37.100370 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df371a67-244b-4e3a-a322-37f1bd32340d None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:37.200332 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df371a67-244b-4e3a-a322-37f1bd32340d None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:37.205390 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15fb3c-c722-4b68-9a6d-a410d47368ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:37.206325 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15fb3c-c722-4b68-9a6d-a410d47368ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:37.206814 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15fb3c-c722-4b68-9a6d-a410d47368ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:37.207276 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15fb3c-c722-4b68-9a6d-a410d47368ee admin admin] PATH_INFO: `/v3/users/aafe6c356bea4d00a532dcea417f2f3e` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:37.208891 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d15fb3c-c722-4b68-9a6d-a410d47368ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=aafe6c356bea4d00a532dcea417f2f3e)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:37.211003 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d15fb3c-c722-4b68-9a6d-a410d47368ee admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:37.296132 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-2d15fb3c-c722-4b68-9a6d-a410d47368ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aafe6c356bea4d00a532dcea417f2f3e'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:37.328825 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-2d15fb3c-c722-4b68-9a6d-a410d47368ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aafe6c356bea4d00a532dcea417f2f3e'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:37.329759 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-2d15fb3c-c722-4b68-9a6d-a410d47368ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aafe6c356bea4d00a532dcea417f2f3e'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:37.330388 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-2d15fb3c-c722-4b68-9a6d-a410d47368ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aafe6c356bea4d00a532dcea417f2f3e'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:37.338563 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-2d15fb3c-c722-4b68-9a6d-a410d47368ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aafe6c356bea4d00a532dcea417f2f3e'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:37.367149 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1379/2754] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:30:37 2022] DELETE /identity/v3/users/aafe6c356bea4d00a532dcea417f2f3e => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:37.372601 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a93b9b-f7dc-4a80-95c4-7ade0859fda0 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:37.373143 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a93b9b-f7dc-4a80-95c4-7ade0859fda0 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:37.373306 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a93b9b-f7dc-4a80-95c4-7ade0859fda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:37.393031 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-b4a93b9b-f7dc-4a80-95c4-7ade0859fda0 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:37.394286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-b4a93b9b-f7dc-4a80-95c4-7ade0859fda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:37.468769 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1376/2755] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:37.479514 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15fb3c-c722-4b68-9a6d-a410d47368ee admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:37.481229 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-2d15fb3c-c722-4b68-9a6d-a410d47368ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:37.545531 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15fb3c-c722-4b68-9a6d-a410d47368ee admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:37.549419 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce745ea-2ebb-4d50-940d-2c73a466ac62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:37.550120 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce745ea-2ebb-4d50-940d-2c73a466ac62 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:37.550419 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce745ea-2ebb-4d50-940d-2c73a466ac62 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:37.550694 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce745ea-2ebb-4d50-940d-2c73a466ac62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:37.561445 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce745ea-2ebb-4d50-940d-2c73a466ac62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:37.562932 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce745ea-2ebb-4d50-940d-2c73a466ac62 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:37.575177 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1380/2756] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:37.902135 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a93b9b-f7dc-4a80-95c4-7ade0859fda0 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:37.903961 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-b4a93b9b-f7dc-4a80-95c4-7ade0859fda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:37.919503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a93b9b-f7dc-4a80-95c4-7ade0859fda0 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:37.924824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf179c-f7ad-4d81-bc6e-e734eda0a4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:37.925897 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf179c-f7ad-4d81-bc6e-e734eda0a4e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:37.926406 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf179c-f7ad-4d81-bc6e-e734eda0a4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:37.926784 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf179c-f7ad-4d81-bc6e-e734eda0a4e4 admin admin] PATH_INFO: `/v3/projects/7034f3ae26504c50bfeb43af8032a477` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:37.928307 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bf179c-f7ad-4d81-bc6e-e734eda0a4e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7034f3ae26504c50bfeb43af8032a477)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:37.930255 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bf179c-f7ad-4d81-bc6e-e734eda0a4e4 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:37.957781 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:37.999377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-67bf179c-f7ad-4d81-bc6e-e734eda0a4e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7034f3ae26504c50bfeb43af8032a477', 'request_id': 'req-67bf179c-f7ad-4d81-bc6e-e734eda0a4e4'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:38.007630 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-67bf179c-f7ad-4d81-bc6e-e734eda0a4e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7034f3ae26504c50bfeb43af8032a477', 'request_id': 'req-67bf179c-f7ad-4d81-bc6e-e734eda0a4e4'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:38.016062 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-67bf179c-f7ad-4d81-bc6e-e734eda0a4e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7034f3ae26504c50bfeb43af8032a477', 'request_id': 'req-67bf179c-f7ad-4d81-bc6e-e734eda0a4e4'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:38.033485 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-67bf179c-f7ad-4d81-bc6e-e734eda0a4e4 admin admin] The token cache is being invalidate because project 7034f3ae26504c50bfeb43af8032a477 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:30:38.033747 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-67bf179c-f7ad-4d81-bc6e-e734eda0a4e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:38.034572 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-67bf179c-f7ad-4d81-bc6e-e734eda0a4e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:38.036428 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1377/2757] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:30:37 2022] DELETE /identity/v3/projects/7034f3ae26504c50bfeb43af8032a477 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:38.040766 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce745ea-2ebb-4d50-940d-2c73a466ac62 service neutron] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:38.051810 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce745ea-2ebb-4d50-940d-2c73a466ac62 service neutron] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:38.069400 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:38.128145 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f3a5d8-918f-4d1c-aa29-796234c14f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:38.128145 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f3a5d8-918f-4d1c-aa29-796234c14f9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:38.128948 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f3a5d8-918f-4d1c-aa29-796234c14f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:38.128948 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f3a5d8-918f-4d1c-aa29-796234c14f9b admin admin] PATH_INFO: `/v3/users/42d753eb491840fcbd5c728c16ba40ee` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:38.129896 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f3a5d8-918f-4d1c-aa29-796234c14f9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=42d753eb491840fcbd5c728c16ba40ee)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:38.131956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f3a5d8-918f-4d1c-aa29-796234c14f9b admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:38.195744 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-55f3a5d8-918f-4d1c-aa29-796234c14f9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42d753eb491840fcbd5c728c16ba40ee'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:38.214352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-55f3a5d8-918f-4d1c-aa29-796234c14f9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42d753eb491840fcbd5c728c16ba40ee'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:38.215259 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-55f3a5d8-918f-4d1c-aa29-796234c14f9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42d753eb491840fcbd5c728c16ba40ee'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:38.216536 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-55f3a5d8-918f-4d1c-aa29-796234c14f9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42d753eb491840fcbd5c728c16ba40ee'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:38.232715 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-55f3a5d8-918f-4d1c-aa29-796234c14f9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42d753eb491840fcbd5c728c16ba40ee'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:38.257941 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1381/2758] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:30:38 2022] DELETE /identity/v3/users/42d753eb491840fcbd5c728c16ba40ee => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:38.598610 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf179c-f7ad-4d81-bc6e-e734eda0a4e4 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:38.601448 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-67bf179c-f7ad-4d81-bc6e-e734eda0a4e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:38.635536 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf179c-f7ad-4d81-bc6e-e734eda0a4e4 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:38.708610 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964f8ce-0ed5-44b7-9364-d4ef1cda76b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:38.709512 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964f8ce-0ed5-44b7-9364-d4ef1cda76b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:38.709797 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964f8ce-0ed5-44b7-9364-d4ef1cda76b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:38.710077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964f8ce-0ed5-44b7-9364-d4ef1cda76b8 admin admin] PATH_INFO: `/v3/projects/b63e4bc464ae43b1a1d0b3dbaaa24edd` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:38.711431 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5964f8ce-0ed5-44b7-9364-d4ef1cda76b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b63e4bc464ae43b1a1d0b3dbaaa24edd)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:38.714656 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5964f8ce-0ed5-44b7-9364-d4ef1cda76b8 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:38.782495 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-5964f8ce-0ed5-44b7-9364-d4ef1cda76b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b63e4bc464ae43b1a1d0b3dbaaa24edd', 'request_id': 'req-5964f8ce-0ed5-44b7-9364-d4ef1cda76b8'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:38.789824 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-5964f8ce-0ed5-44b7-9364-d4ef1cda76b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b63e4bc464ae43b1a1d0b3dbaaa24edd', 'request_id': 'req-5964f8ce-0ed5-44b7-9364-d4ef1cda76b8'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:38.797924 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-5964f8ce-0ed5-44b7-9364-d4ef1cda76b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b63e4bc464ae43b1a1d0b3dbaaa24edd', 'request_id': 'req-5964f8ce-0ed5-44b7-9364-d4ef1cda76b8'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:38.814658 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-5964f8ce-0ed5-44b7-9364-d4ef1cda76b8 admin admin] The token cache is being invalidate because project b63e4bc464ae43b1a1d0b3dbaaa24edd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:30:38.814942 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-5964f8ce-0ed5-44b7-9364-d4ef1cda76b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:38.815683 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-5964f8ce-0ed5-44b7-9364-d4ef1cda76b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:38.817283 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1378/2759] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:30:38 2022] DELETE /identity/v3/projects/b63e4bc464ae43b1a1d0b3dbaaa24edd => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:38.822967 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b9960-1109-4912-8649-ebf31c99b5b6 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:38.823626 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b9960-1109-4912-8649-ebf31c99b5b6 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:38.824091 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b9960-1109-4912-8649-ebf31c99b5b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:38.851457 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-539b9960-1109-4912-8649-ebf31c99b5b6 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:38.853693 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-539b9960-1109-4912-8649-ebf31c99b5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:38.926651 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1382/2760] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:38.933631 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d719b-803b-4474-81f7-e15c7e1b6efa None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:38.933922 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d719b-803b-4474-81f7-e15c7e1b6efa None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:38.934376 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d719b-803b-4474-81f7-e15c7e1b6efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:38.957009 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-a20d719b-803b-4474-81f7-e15c7e1b6efa None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:38.958358 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-a20d719b-803b-4474-81f7-e15c7e1b6efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:38.967504 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1379/2761] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:38.972308 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b9960-1109-4912-8649-ebf31c99b5b6 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:38.984956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b9960-1109-4912-8649-ebf31c99b5b6 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:38.992029 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c9d9c-e22a-48e4-a0ad-ba738aed8c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:38.993019 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c9d9c-e22a-48e4-a0ad-ba738aed8c69 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:38.993435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c9d9c-e22a-48e4-a0ad-ba738aed8c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:38.993991 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c9d9c-e22a-48e4-a0ad-ba738aed8c69 admin admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:38.995195 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c9d9c-e22a-48e4-a0ad-ba738aed8c69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:30:38.995676 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c9d9c-e22a-48e4-a0ad-ba738aed8c69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:38.998442 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c9d9c-e22a-48e4-a0ad-ba738aed8c69 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:39.016177 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1383/2762] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 15:30:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:30:39.020924 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d719b-803b-4474-81f7-e15c7e1b6efa None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:39.031187 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d719b-803b-4474-81f7-e15c7e1b6efa None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:39.037261 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d753c0-bc2a-487b-a6d2-4d74f6abe7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:39.037926 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d753c0-bc2a-487b-a6d2-4d74f6abe7d3 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:39.038281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d753c0-bc2a-487b-a6d2-4d74f6abe7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:39.038707 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d753c0-bc2a-487b-a6d2-4d74f6abe7d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:39.039847 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d753c0-bc2a-487b-a6d2-4d74f6abe7d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:39.042273 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d753c0-bc2a-487b-a6d2-4d74f6abe7d3 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:39.086258 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1380/2763] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:30:39 2022] POST /identity/v3/projects => generated 365 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:30:39.092714 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c9d9c-e22a-48e4-a0ad-ba738aed8c69 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:39.106326 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c9d9c-e22a-48e4-a0ad-ba738aed8c69 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:39.133125 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:39.181732 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc2bcb4-2d5a-4eb0-ba41-023c4aed1656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:39.182506 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2bcb4-2d5a-4eb0-ba41-023c4aed1656 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:39.182817 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2bcb4-2d5a-4eb0-ba41-023c4aed1656 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:39.183124 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2bcb4-2d5a-4eb0-ba41-023c4aed1656 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:39.183877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc2bcb4-2d5a-4eb0-ba41-023c4aed1656 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:39.186039 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc2bcb4-2d5a-4eb0-ba41-023c4aed1656 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:39.212083 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1384/2764] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:30:39 2022] POST /identity/v3/users => generated 283 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:30:39.218552 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d753c0-bc2a-487b-a6d2-4d74f6abe7d3 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:39.233697 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d753c0-bc2a-487b-a6d2-4d74f6abe7d3 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:39.244672 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78618f73-b812-4c06-85fb-3200e59701f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:39.245409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78618f73-b812-4c06-85fb-3200e59701f5 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:39.245743 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78618f73-b812-4c06-85fb-3200e59701f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:39.246079 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78618f73-b812-4c06-85fb-3200e59701f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:39.247206 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78618f73-b812-4c06-85fb-3200e59701f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:39.249341 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78618f73-b812-4c06-85fb-3200e59701f5 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:39.252055 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:39.265626 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1381/2765] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:30:39.270587 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc2bcb4-2d5a-4eb0-ba41-023c4aed1656 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:39.287692 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc2bcb4-2d5a-4eb0-ba41-023c4aed1656 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:39.293613 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a3cb0-a27e-46de-969a-2003cd39362f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:39.294346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a3cb0-a27e-46de-969a-2003cd39362f admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:39.294659 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a3cb0-a27e-46de-969a-2003cd39362f admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:39.294941 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a3cb0-a27e-46de-969a-2003cd39362f admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:39.295738 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a3cb0-a27e-46de-969a-2003cd39362f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:39.297526 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a3cb0-a27e-46de-969a-2003cd39362f admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:39.311915 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1385/2766] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:30:39.316706 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78618f73-b812-4c06-85fb-3200e59701f5 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:39.328302 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78618f73-b812-4c06-85fb-3200e59701f5 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:39.334370 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cdbcd7-1209-4546-900c-85b90942792c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:39.335408 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdbcd7-1209-4546-900c-85b90942792c admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:39.335689 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdbcd7-1209-4546-900c-85b90942792c admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:39.335968 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdbcd7-1209-4546-900c-85b90942792c admin admin] PATH_INFO: `/v3/projects/977152a3f6f84518a472677ccc5e6156/users/f9d89a78541d42ab9c535dfd0e7ae609/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:39.353711 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cdbcd7-1209-4546-900c-85b90942792c admin admin] RBAC: Authorizing `identity:create_grant(project_id=977152a3f6f84518a472677ccc5e6156, user_id=f9d89a78541d42ab9c535dfd0e7ae609, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:39.356494 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cdbcd7-1209-4546-900c-85b90942792c admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:39.373324 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1382/2767] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:30:39 2022] PUT /identity/v3/projects/977152a3f6f84518a472677ccc5e6156/users/f9d89a78541d42ab9c535dfd0e7ae609/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:39.386641 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b97dd1-93d6-449d-8898-bfc8516ebef4 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:39.386894 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b97dd1-93d6-449d-8898-bfc8516ebef4 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:39.387251 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b97dd1-93d6-449d-8898-bfc8516ebef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:39.417030 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-29b97dd1-93d6-449d-8898-bfc8516ebef4 None None] MFA Rules not processed for user `f9d89a78541d42ab9c535dfd0e7ae609`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:39.418269 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-29b97dd1-93d6-449d-8898-bfc8516ebef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:39.493238 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1386/2768] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:39 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:39.498082 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cdbcd7-1209-4546-900c-85b90942792c admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:39.568063 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cdbcd7-1209-4546-900c-85b90942792c admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:39.573213 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acffe286-e542-4c62-a3f9-c4a4b4697274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:39.573893 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acffe286-e542-4c62-a3f9-c4a4b4697274 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:39.574249 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acffe286-e542-4c62-a3f9-c4a4b4697274 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:39.574540 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acffe286-e542-4c62-a3f9-c4a4b4697274 admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:39.575272 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acffe286-e542-4c62-a3f9-c4a4b4697274 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:39.577609 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acffe286-e542-4c62-a3f9-c4a4b4697274 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:39.607739 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1383/2769] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:30:39 2022] POST /identity/v3/projects => generated 367 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:30:39.612585 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b97dd1-93d6-449d-8898-bfc8516ebef4 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:39.625061 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b97dd1-93d6-449d-8898-bfc8516ebef4 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:39.703756 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a6866-06d2-490e-a4d2-6f33f73be989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:39.704513 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a6866-06d2-490e-a4d2-6f33f73be989 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:39.704778 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a6866-06d2-490e-a4d2-6f33f73be989 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:39.705042 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a6866-06d2-490e-a4d2-6f33f73be989 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:39.705803 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28a6866-06d2-490e-a4d2-6f33f73be989 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:39.707958 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28a6866-06d2-490e-a4d2-6f33f73be989 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:39.743187 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1387/2770] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:30:39 2022] POST /identity/v3/users => generated 290 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:30:39.747825 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acffe286-e542-4c62-a3f9-c4a4b4697274 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:39.765769 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acffe286-e542-4c62-a3f9-c4a4b4697274 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:39.771846 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117aa50-f89f-4a8b-8e85-58c91967eefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:39.772546 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117aa50-f89f-4a8b-8e85-58c91967eefc admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:39.772831 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117aa50-f89f-4a8b-8e85-58c91967eefc admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:39.773134 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117aa50-f89f-4a8b-8e85-58c91967eefc admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:39.773947 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117aa50-f89f-4a8b-8e85-58c91967eefc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:39.775722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117aa50-f89f-4a8b-8e85-58c91967eefc admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:39.788405 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1384/2771] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:30:39.792791 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a6866-06d2-490e-a4d2-6f33f73be989 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:39.805449 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a6866-06d2-490e-a4d2-6f33f73be989 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:39.811864 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d030b-69d7-4b93-8d20-ba3c5321d398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:39.812842 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d030b-69d7-4b93-8d20-ba3c5321d398 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:39.813140 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d030b-69d7-4b93-8d20-ba3c5321d398 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:39.813402 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d030b-69d7-4b93-8d20-ba3c5321d398 admin admin] PATH_INFO: `/v3/projects/f00ceef63ab34f90a45864725f75a1a3/users/a19b5ea9f4ac4dd0ad311f35187161b7/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:39.835792 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d030b-69d7-4b93-8d20-ba3c5321d398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f00ceef63ab34f90a45864725f75a1a3, user_id=a19b5ea9f4ac4dd0ad311f35187161b7, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:39.839105 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d030b-69d7-4b93-8d20-ba3c5321d398 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:39.850909 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:39.860799 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1388/2772] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:30:39 2022] PUT /identity/v3/projects/f00ceef63ab34f90a45864725f75a1a3/users/a19b5ea9f4ac4dd0ad311f35187161b7/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:39.865234 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117aa50-f89f-4a8b-8e85-58c91967eefc admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:39.876054 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117aa50-f89f-4a8b-8e85-58c91967eefc admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:39.963353 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba270ec-1185-4408-9554-8a9991c814c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:39.964119 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba270ec-1185-4408-9554-8a9991c814c7 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:39.964407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba270ec-1185-4408-9554-8a9991c814c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:39.964620 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba270ec-1185-4408-9554-8a9991c814c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:39.965348 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba270ec-1185-4408-9554-8a9991c814c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:39.966979 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba270ec-1185-4408-9554-8a9991c814c7 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:39.978487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1385/2773] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:30:39.982663 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d030b-69d7-4b93-8d20-ba3c5321d398 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:39.993967 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d030b-69d7-4b93-8d20-ba3c5321d398 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:39.999198 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1680dae6-3848-416c-a02a-b8c25fc6a8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:40.000152 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680dae6-3848-416c-a02a-b8c25fc6a8dd admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:40.001158 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680dae6-3848-416c-a02a-b8c25fc6a8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:40.001158 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680dae6-3848-416c-a02a-b8c25fc6a8dd admin admin] PATH_INFO: `/v3/projects/f00ceef63ab34f90a45864725f75a1a3/users/a19b5ea9f4ac4dd0ad311f35187161b7/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:40.010068 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1680dae6-3848-416c-a02a-b8c25fc6a8dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f00ceef63ab34f90a45864725f75a1a3, user_id=a19b5ea9f4ac4dd0ad311f35187161b7, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:40.015820 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1680dae6-3848-416c-a02a-b8c25fc6a8dd admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:40.030684 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1389/2774] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:30:39 2022] PUT /identity/v3/projects/f00ceef63ab34f90a45864725f75a1a3/users/a19b5ea9f4ac4dd0ad311f35187161b7/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:40.041433 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fa193-9190-48a6-b0c5-1b165cb6f6e6 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:40.041720 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fa193-9190-48a6-b0c5-1b165cb6f6e6 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:40.042011 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fa193-9190-48a6-b0c5-1b165cb6f6e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:40.071284 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-248fa193-9190-48a6-b0c5-1b165cb6f6e6 None None] MFA Rules not processed for user `a19b5ea9f4ac4dd0ad311f35187161b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:40.072578 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-248fa193-9190-48a6-b0c5-1b165cb6f6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:40.115701 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:40.163407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1386/2775] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:40 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:40.176831 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1680dae6-3848-416c-a02a-b8c25fc6a8dd admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:40.177476 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-1680dae6-3848-416c-a02a-b8c25fc6a8dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:40.246714 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1680dae6-3848-416c-a02a-b8c25fc6a8dd admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:40.251220 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47389d08-0887-4c79-b96d-f9a7bf3606bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:40.251914 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47389d08-0887-4c79-b96d-f9a7bf3606bd service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:40.252171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47389d08-0887-4c79-b96d-f9a7bf3606bd service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:40.252441 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47389d08-0887-4c79-b96d-f9a7bf3606bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:40.263412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47389d08-0887-4c79-b96d-f9a7bf3606bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:40.264864 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47389d08-0887-4c79-b96d-f9a7bf3606bd service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:40.275069 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1390/2776] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:40 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:40.365490 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fa193-9190-48a6-b0c5-1b165cb6f6e6 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:40.376821 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fa193-9190-48a6-b0c5-1b165cb6f6e6 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:40.381436 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292de76b-e969-4a46-9887-2e9e718fb83a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:40.382173 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292de76b-e969-4a46-9887-2e9e718fb83a service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:40.382446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292de76b-e969-4a46-9887-2e9e718fb83a service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:40.382794 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292de76b-e969-4a46-9887-2e9e718fb83a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:40.458376 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292de76b-e969-4a46-9887-2e9e718fb83a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:40.460803 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292de76b-e969-4a46-9887-2e9e718fb83a service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:40.473338 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1387/2777] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:40 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:42.561036 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47389d08-0887-4c79-b96d-f9a7bf3606bd service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:42.562658 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-47389d08-0887-4c79-b96d-f9a7bf3606bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:42.623688 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47389d08-0887-4c79-b96d-f9a7bf3606bd service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:42.628127 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297caced-a5fc-4914-ab55-8bb4f512392d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:42.628833 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297caced-a5fc-4914-ab55-8bb4f512392d service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:42.629115 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297caced-a5fc-4914-ab55-8bb4f512392d service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:42.629389 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297caced-a5fc-4914-ab55-8bb4f512392d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:42.631847 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-297caced-a5fc-4914-ab55-8bb4f512392d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:42.704537 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297caced-a5fc-4914-ab55-8bb4f512392d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:42.706033 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297caced-a5fc-4914-ab55-8bb4f512392d service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:42.717587 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1391/2778] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:45.540734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292de76b-e969-4a46-9887-2e9e718fb83a service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:45.552527 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292de76b-e969-4a46-9887-2e9e718fb83a service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:45.559144 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5d008-ea2e-4144-b367-6450a13ac7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:45.560012 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5d008-ea2e-4144-b367-6450a13ac7c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:45.560167 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5d008-ea2e-4144-b367-6450a13ac7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:45.560448 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5d008-ea2e-4144-b367-6450a13ac7c9 admin admin] PATH_INFO: `/v3/users/f9d89a78541d42ab9c535dfd0e7ae609` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:45.567290 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5d008-ea2e-4144-b367-6450a13ac7c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9d89a78541d42ab9c535dfd0e7ae609)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:45.569853 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5d008-ea2e-4144-b367-6450a13ac7c9 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:45.647015 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-0df5d008-ea2e-4144-b367-6450a13ac7c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9d89a78541d42ab9c535dfd0e7ae609'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:45.674095 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-0df5d008-ea2e-4144-b367-6450a13ac7c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9d89a78541d42ab9c535dfd0e7ae609'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:45.675294 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-0df5d008-ea2e-4144-b367-6450a13ac7c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9d89a78541d42ab9c535dfd0e7ae609'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:45.676294 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-0df5d008-ea2e-4144-b367-6450a13ac7c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9d89a78541d42ab9c535dfd0e7ae609'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:45.689955 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-0df5d008-ea2e-4144-b367-6450a13ac7c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9d89a78541d42ab9c535dfd0e7ae609'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:45.709880 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1388/2779] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:30:45 2022] DELETE /identity/v3/users/f9d89a78541d42ab9c535dfd0e7ae609 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:45.714804 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08b259-9d28-4e12-a320-4bee625b32f3 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:45.715370 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08b259-9d28-4e12-a320-4bee625b32f3 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:45.715483 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08b259-9d28-4e12-a320-4bee625b32f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:45.749631 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-0a08b259-9d28-4e12-a320-4bee625b32f3 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:45.750886 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-0a08b259-9d28-4e12-a320-4bee625b32f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:45.805026 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:45.861965 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1392/2780] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:30:45.869493 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5d008-ea2e-4144-b367-6450a13ac7c9 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:45.870986 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-0df5d008-ea2e-4144-b367-6450a13ac7c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:45.927153 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:45.950054 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5d008-ea2e-4144-b367-6450a13ac7c9 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:45.965641 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc5ba3-8872-4cc4-a5af-61d094d33a9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:45.966729 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc5ba3-8872-4cc4-a5af-61d094d33a9d service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:45.967183 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc5ba3-8872-4cc4-a5af-61d094d33a9d service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:45.967643 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc5ba3-8872-4cc4-a5af-61d094d33a9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:45.985347 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdc5ba3-8872-4cc4-a5af-61d094d33a9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:45.986823 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdc5ba3-8872-4cc4-a5af-61d094d33a9d service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:45.998209 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1389/2781] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:46.265013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08b259-9d28-4e12-a320-4bee625b32f3 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:46.266790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-0a08b259-9d28-4e12-a320-4bee625b32f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:46.282933 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08b259-9d28-4e12-a320-4bee625b32f3 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:46.287919 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35cd1f0-a19d-420d-af45-097c9478481a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:46.288824 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35cd1f0-a19d-420d-af45-097c9478481a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:46.289168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35cd1f0-a19d-420d-af45-097c9478481a admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:46.289495 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35cd1f0-a19d-420d-af45-097c9478481a admin admin] PATH_INFO: `/v3/users/e3574b5b8f3241b192b0b810122d4769` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:46.290846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35cd1f0-a19d-420d-af45-097c9478481a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3574b5b8f3241b192b0b810122d4769)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:46.292901 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35cd1f0-a19d-420d-af45-097c9478481a admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:46.312996 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc5ba3-8872-4cc4-a5af-61d094d33a9d service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:46.314591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-9bdc5ba3-8872-4cc4-a5af-61d094d33a9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:46.330859 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc5ba3-8872-4cc4-a5af-61d094d33a9d service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:46.336469 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f5e100-191f-4a92-8f06-a1db985ee805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:46.337687 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5e100-191f-4a92-8f06-a1db985ee805 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:46.338178 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5e100-191f-4a92-8f06-a1db985ee805 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:46.338559 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5e100-191f-4a92-8f06-a1db985ee805 admin admin] PATH_INFO: `/v3/projects/977152a3f6f84518a472677ccc5e6156` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:46.341216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f5e100-191f-4a92-8f06-a1db985ee805 admin admin] RBAC: Authorizing `identity:delete_project(project_id=977152a3f6f84518a472677ccc5e6156)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:46.343809 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f5e100-191f-4a92-8f06-a1db985ee805 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:46.356945 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-c35cd1f0-a19d-420d-af45-097c9478481a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3574b5b8f3241b192b0b810122d4769'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:46.369410 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-c35cd1f0-a19d-420d-af45-097c9478481a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3574b5b8f3241b192b0b810122d4769'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:46.370306 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-c35cd1f0-a19d-420d-af45-097c9478481a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3574b5b8f3241b192b0b810122d4769'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:46.371038 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-c35cd1f0-a19d-420d-af45-097c9478481a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3574b5b8f3241b192b0b810122d4769'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:46.378947 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-c35cd1f0-a19d-420d-af45-097c9478481a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3574b5b8f3241b192b0b810122d4769'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:46.393584 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1393/2782] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:30:46 2022] DELETE /identity/v3/users/e3574b5b8f3241b192b0b810122d4769 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:46.400899 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-24f5e100-191f-4a92-8f06-a1db985ee805 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '977152a3f6f84518a472677ccc5e6156', 'request_id': 'req-24f5e100-191f-4a92-8f06-a1db985ee805'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:46.408276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-24f5e100-191f-4a92-8f06-a1db985ee805 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '977152a3f6f84518a472677ccc5e6156', 'request_id': 'req-24f5e100-191f-4a92-8f06-a1db985ee805'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:46.415248 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-24f5e100-191f-4a92-8f06-a1db985ee805 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '977152a3f6f84518a472677ccc5e6156', 'request_id': 'req-24f5e100-191f-4a92-8f06-a1db985ee805'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:46.431716 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-24f5e100-191f-4a92-8f06-a1db985ee805 admin admin] The token cache is being invalidate because project 977152a3f6f84518a472677ccc5e6156 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:30:46.432001 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-24f5e100-191f-4a92-8f06-a1db985ee805 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:46.433245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-24f5e100-191f-4a92-8f06-a1db985ee805 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:46.434343 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1390/2783] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:30:46 2022] DELETE /identity/v3/projects/977152a3f6f84518a472677ccc5e6156 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:46.439099 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35cd1f0-a19d-420d-af45-097c9478481a admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:46.440247 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-c35cd1f0-a19d-420d-af45-097c9478481a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:46.517101 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35cd1f0-a19d-420d-af45-097c9478481a admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:46.523210 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5e15f-2a02-4148-9a42-bc40a740a444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:46.524178 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5e15f-2a02-4148-9a42-bc40a740a444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:46.524474 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5e15f-2a02-4148-9a42-bc40a740a444 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:46.524833 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5e15f-2a02-4148-9a42-bc40a740a444 admin admin] PATH_INFO: `/v3/users/a19b5ea9f4ac4dd0ad311f35187161b7` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:46.526079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5e15f-2a02-4148-9a42-bc40a740a444 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a19b5ea9f4ac4dd0ad311f35187161b7)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:46.528080 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5e15f-2a02-4148-9a42-bc40a740a444 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:46.585228 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-e0f5e15f-2a02-4148-9a42-bc40a740a444 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a19b5ea9f4ac4dd0ad311f35187161b7'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:46.590862 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:46.593235 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f5e100-191f-4a92-8f06-a1db985ee805 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:46.595479 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-24f5e100-191f-4a92-8f06-a1db985ee805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:46.614549 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-e0f5e15f-2a02-4148-9a42-bc40a740a444 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a19b5ea9f4ac4dd0ad311f35187161b7'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:46.615950 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-e0f5e15f-2a02-4148-9a42-bc40a740a444 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a19b5ea9f4ac4dd0ad311f35187161b7'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:46.616605 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-e0f5e15f-2a02-4148-9a42-bc40a740a444 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a19b5ea9f4ac4dd0ad311f35187161b7'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:46.624633 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-e0f5e15f-2a02-4148-9a42-bc40a740a444 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a19b5ea9f4ac4dd0ad311f35187161b7'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:46.641494 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1394/2784] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:30:46 2022] DELETE /identity/v3/users/a19b5ea9f4ac4dd0ad311f35187161b7 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:46.677362 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f5e100-191f-4a92-8f06-a1db985ee805 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:46.740771 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646e493-f234-4ca7-b8a5-8833a645daa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:46.741502 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646e493-f234-4ca7-b8a5-8833a645daa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:46.741787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646e493-f234-4ca7-b8a5-8833a645daa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:46.742186 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646e493-f234-4ca7-b8a5-8833a645daa3 admin admin] PATH_INFO: `/v3/projects/5d984345e0d340818c9030f70bace6d3` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:46.743748 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e646e493-f234-4ca7-b8a5-8833a645daa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d984345e0d340818c9030f70bace6d3)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:46.746399 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e646e493-f234-4ca7-b8a5-8833a645daa3 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:46.763655 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:46.809921 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-e646e493-f234-4ca7-b8a5-8833a645daa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d984345e0d340818c9030f70bace6d3', 'request_id': 'req-e646e493-f234-4ca7-b8a5-8833a645daa3'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:46.817585 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-e646e493-f234-4ca7-b8a5-8833a645daa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d984345e0d340818c9030f70bace6d3', 'request_id': 'req-e646e493-f234-4ca7-b8a5-8833a645daa3'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:46.825744 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-e646e493-f234-4ca7-b8a5-8833a645daa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d984345e0d340818c9030f70bace6d3', 'request_id': 'req-e646e493-f234-4ca7-b8a5-8833a645daa3'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:46.841275 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-e646e493-f234-4ca7-b8a5-8833a645daa3 admin admin] The token cache is being invalidate because project 5d984345e0d340818c9030f70bace6d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:30:46.841774 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-e646e493-f234-4ca7-b8a5-8833a645daa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:46.842609 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-e646e493-f234-4ca7-b8a5-8833a645daa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:46.844612 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1391/2785] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:30:46 2022] DELETE /identity/v3/projects/5d984345e0d340818c9030f70bace6d3 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:46.849941 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a710dcd-391e-4947-820a-ef560e1738f3 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:46.850463 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a710dcd-391e-4947-820a-ef560e1738f3 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:46.850797 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a710dcd-391e-4947-820a-ef560e1738f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:46.870715 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-1a710dcd-391e-4947-820a-ef560e1738f3 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:46.872021 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-1a710dcd-391e-4947-820a-ef560e1738f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:46.932081 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646e493-f234-4ca7-b8a5-8833a645daa3 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:46.934315 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e646e493-f234-4ca7-b8a5-8833a645daa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:46.935140 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1395/2786] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:46.943510 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49974c-8a48-4893-a2b8-cfa3664f68a8 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:46.943816 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49974c-8a48-4893-a2b8-cfa3664f68a8 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:46.944151 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49974c-8a48-4893-a2b8-cfa3664f68a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:46.958540 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646e493-f234-4ca7-b8a5-8833a645daa3 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:46.964603 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9016e-129d-4891-8d13-685cb23cd301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:46.965613 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9016e-129d-4891-8d13-685cb23cd301 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:46.965901 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9016e-129d-4891-8d13-685cb23cd301 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:46.966166 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-0f49974c-8a48-4893-a2b8-cfa3664f68a8 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:46.966286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9016e-129d-4891-8d13-685cb23cd301 admin admin] PATH_INFO: `/v3/projects/f00ceef63ab34f90a45864725f75a1a3` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:46.967467 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-0f49974c-8a48-4893-a2b8-cfa3664f68a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:46.967691 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b9016e-129d-4891-8d13-685cb23cd301 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f00ceef63ab34f90a45864725f75a1a3)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:46.969641 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b9016e-129d-4891-8d13-685cb23cd301 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:46.976691 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1396/2787] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:46.982525 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49974c-8a48-4893-a2b8-cfa3664f68a8 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:46.993073 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49974c-8a48-4893-a2b8-cfa3664f68a8 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:46.997868 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40677af7-f58f-4a75-830c-83a2f5871dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:46.998565 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40677af7-f58f-4a75-830c-83a2f5871dba admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:46.998844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40677af7-f58f-4a75-830c-83a2f5871dba admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:46.999114 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40677af7-f58f-4a75-830c-83a2f5871dba admin admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:46.999886 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40677af7-f58f-4a75-830c-83a2f5871dba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:30:47.000231 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40677af7-f58f-4a75-830c-83a2f5871dba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:47.002075 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40677af7-f58f-4a75-830c-83a2f5871dba admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:47.015086 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1397/2788] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 15:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:30:47.019643 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40677af7-f58f-4a75-830c-83a2f5871dba admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:47.022751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-c5b9016e-129d-4891-8d13-685cb23cd301 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f00ceef63ab34f90a45864725f75a1a3', 'request_id': 'req-c5b9016e-129d-4891-8d13-685cb23cd301'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:47.029450 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-c5b9016e-129d-4891-8d13-685cb23cd301 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f00ceef63ab34f90a45864725f75a1a3', 'request_id': 'req-c5b9016e-129d-4891-8d13-685cb23cd301'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:47.031079 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40677af7-f58f-4a75-830c-83a2f5871dba admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:47.036382 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-c5b9016e-129d-4891-8d13-685cb23cd301 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f00ceef63ab34f90a45864725f75a1a3', 'request_id': 'req-c5b9016e-129d-4891-8d13-685cb23cd301'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:47.055767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-c5b9016e-129d-4891-8d13-685cb23cd301 admin admin] The token cache is being invalidate because project f00ceef63ab34f90a45864725f75a1a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:30:47.056231 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-c5b9016e-129d-4891-8d13-685cb23cd301 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:47.057231 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-c5b9016e-129d-4891-8d13-685cb23cd301 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:47.060022 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1392/2789] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:30:46 2022] DELETE /identity/v3/projects/f00ceef63ab34f90a45864725f75a1a3 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:47.067265 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5674bfe-2e2d-4401-b3e9-67c3df28663a None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:47.067579 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5674bfe-2e2d-4401-b3e9-67c3df28663a None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:47.067842 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5674bfe-2e2d-4401-b3e9-67c3df28663a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:47.087965 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-c5674bfe-2e2d-4401-b3e9-67c3df28663a None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:47.089261 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-c5674bfe-2e2d-4401-b3e9-67c3df28663a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:47.095809 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f49e38-f29e-4416-9113-6bdddf80eabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:47.096863 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f49e38-f29e-4416-9113-6bdddf80eabb admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:47.097155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f49e38-f29e-4416-9113-6bdddf80eabb admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:47.097441 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f49e38-f29e-4416-9113-6bdddf80eabb admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:47.098189 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f49e38-f29e-4416-9113-6bdddf80eabb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:47.100264 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f49e38-f29e-4416-9113-6bdddf80eabb admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:47.137108 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1398/2790] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:30:47 2022] POST /identity/v3/projects => generated 365 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:30:47.145274 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f49e38-f29e-4416-9113-6bdddf80eabb admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:47.147116 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-c4f49e38-f29e-4416-9113-6bdddf80eabb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:47.169691 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1393/2791] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:30:47.181893 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41f640-7655-42bd-8ea4-cbcd248ea5a0 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:47.182236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41f640-7655-42bd-8ea4-cbcd248ea5a0 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:47.182472 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41f640-7655-42bd-8ea4-cbcd248ea5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:47.203469 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:47.205925 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-ff41f640-7655-42bd-8ea4-cbcd248ea5a0 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:47.207246 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-ff41f640-7655-42bd-8ea4-cbcd248ea5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:47.236980 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f49e38-f29e-4416-9113-6bdddf80eabb admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:47.237238 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:47.244337 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b931907-5054-4c95-a8aa-30ef1104362c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:47.245074 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b931907-5054-4c95-a8aa-30ef1104362c admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:47.245375 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b931907-5054-4c95-a8aa-30ef1104362c admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:47.245654 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b931907-5054-4c95-a8aa-30ef1104362c admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:47.246373 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b931907-5054-4c95-a8aa-30ef1104362c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:47.248630 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b931907-5054-4c95-a8aa-30ef1104362c admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:47.274543 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1399/2792] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:30:47 2022] POST /identity/v3/users => generated 283 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:30:47.280408 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b931907-5054-4c95-a8aa-30ef1104362c admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:47.291874 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1394/2793] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:47.292725 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b931907-5054-4c95-a8aa-30ef1104362c admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:47.296159 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41f640-7655-42bd-8ea4-cbcd248ea5a0 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:47.299280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4708f9b5-e721-4c93-9475-bc9cd1294588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:47.300128 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4708f9b5-e721-4c93-9475-bc9cd1294588 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:47.300472 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4708f9b5-e721-4c93-9475-bc9cd1294588 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:47.300991 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4708f9b5-e721-4c93-9475-bc9cd1294588 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:47.302268 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4708f9b5-e721-4c93-9475-bc9cd1294588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:47.303921 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4708f9b5-e721-4c93-9475-bc9cd1294588 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:47.306552 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41f640-7655-42bd-8ea4-cbcd248ea5a0 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:47.315399 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827251a-ab35-493b-9dab-aac46096a1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:47.318992 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1400/2794] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:30:47.319134 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827251a-ab35-493b-9dab-aac46096a1f4 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:47.319134 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827251a-ab35-493b-9dab-aac46096a1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:47.319134 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827251a-ab35-493b-9dab-aac46096a1f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:47.319134 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e827251a-ab35-493b-9dab-aac46096a1f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:30:47.319134 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e827251a-ab35-493b-9dab-aac46096a1f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:47.320590 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e827251a-ab35-493b-9dab-aac46096a1f4 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:47.323945 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4708f9b5-e721-4c93-9475-bc9cd1294588 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:47.335934 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4708f9b5-e721-4c93-9475-bc9cd1294588 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:47.340432 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1395/2795] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 15:30:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:30:47.342179 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827251a-ab35-493b-9dab-aac46096a1f4 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:47.344198 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a6bf80-bb43-4c1a-9ccb-e2ed28c72105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:47.345035 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6bf80-bb43-4c1a-9ccb-e2ed28c72105 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:47.345331 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6bf80-bb43-4c1a-9ccb-e2ed28c72105 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:47.345625 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6bf80-bb43-4c1a-9ccb-e2ed28c72105 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:47.346508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a6bf80-bb43-4c1a-9ccb-e2ed28c72105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:47.348175 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a6bf80-bb43-4c1a-9ccb-e2ed28c72105 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:47.353172 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827251a-ab35-493b-9dab-aac46096a1f4 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:47.361719 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1401/2796] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:30:47.365186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a6bf80-bb43-4c1a-9ccb-e2ed28c72105 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:47.368954 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7e649-09e0-4921-bae6-7de69d740dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:47.369916 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7e649-09e0-4921-bae6-7de69d740dbf admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:47.370414 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7e649-09e0-4921-bae6-7de69d740dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:47.370769 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7e649-09e0-4921-bae6-7de69d740dbf admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:47.371864 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b7e649-09e0-4921-bae6-7de69d740dbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:47.374895 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b7e649-09e0-4921-bae6-7de69d740dbf admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:47.379603 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a6bf80-bb43-4c1a-9ccb-e2ed28c72105 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:47.388611 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3bd26-07f7-4c17-9637-57789121ee60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:47.389580 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3bd26-07f7-4c17-9637-57789121ee60 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:47.389877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3bd26-07f7-4c17-9637-57789121ee60 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:47.390155 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3bd26-07f7-4c17-9637-57789121ee60 admin admin] PATH_INFO: `/v3/projects/ac0ccd08607644ee89c9c8e23f0ae369/users/9d5a553e2aa2488581ccefcfbaf70ca3/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:47.408196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3bd26-07f7-4c17-9637-57789121ee60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac0ccd08607644ee89c9c8e23f0ae369, user_id=9d5a553e2aa2488581ccefcfbaf70ca3, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:47.411405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3bd26-07f7-4c17-9637-57789121ee60 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:47.413614 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1396/2797] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:30:47 2022] POST /identity/v3/projects => generated 361 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:30:47.418841 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7e649-09e0-4921-bae6-7de69d740dbf admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:47.434867 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1402/2798] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:30:47 2022] PUT /identity/v3/projects/ac0ccd08607644ee89c9c8e23f0ae369/users/9d5a553e2aa2488581ccefcfbaf70ca3/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:30:47.452174 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e76f2b-293d-4a23-9805-a04edf4e26bc None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:47.452462 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e76f2b-293d-4a23-9805-a04edf4e26bc None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:47.452767 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e76f2b-293d-4a23-9805-a04edf4e26bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:47.490245 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-82e76f2b-293d-4a23-9805-a04edf4e26bc None None] MFA Rules not processed for user `9d5a553e2aa2488581ccefcfbaf70ca3`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:47.491638 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-82e76f2b-293d-4a23-9805-a04edf4e26bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:47.524581 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7e649-09e0-4921-bae6-7de69d740dbf admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:47.563856 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1403/2799] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:47 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:47.568492 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e76f2b-293d-4a23-9805-a04edf4e26bc None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:47.590533 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05df3ee-8801-4d6d-8731-3d514a9dcd30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:47.591403 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05df3ee-8801-4d6d-8731-3d514a9dcd30 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:47.591692 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05df3ee-8801-4d6d-8731-3d514a9dcd30 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:47.591988 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05df3ee-8801-4d6d-8731-3d514a9dcd30 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:47.592794 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05df3ee-8801-4d6d-8731-3d514a9dcd30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:47.594825 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05df3ee-8801-4d6d-8731-3d514a9dcd30 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:47.622796 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1397/2800] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:30:47 2022] POST /identity/v3/users => generated 281 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:30:47.623882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05df3ee-8801-4d6d-8731-3d514a9dcd30 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:47.635893 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05df3ee-8801-4d6d-8731-3d514a9dcd30 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:47.640780 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d012ec-9b05-4205-949c-28df1bb446bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:47.641464 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d012ec-9b05-4205-949c-28df1bb446bf admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:47.641747 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d012ec-9b05-4205-949c-28df1bb446bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:47.642021 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d012ec-9b05-4205-949c-28df1bb446bf admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:47.642889 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d012ec-9b05-4205-949c-28df1bb446bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:47.643365 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e76f2b-293d-4a23-9805-a04edf4e26bc None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:47.644806 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d012ec-9b05-4205-949c-28df1bb446bf admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:47.650206 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5be5b-7266-41f4-9088-93833c889d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:47.650963 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5be5b-7266-41f4-9088-93833c889d3d admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:47.651304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5be5b-7266-41f4-9088-93833c889d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:47.651588 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5be5b-7266-41f4-9088-93833c889d3d admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:47.652303 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5be5b-7266-41f4-9088-93833c889d3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:47.654341 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5be5b-7266-41f4-9088-93833c889d3d admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:47.657238 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1398/2801] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:30:47.661118 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d012ec-9b05-4205-949c-28df1bb446bf admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:47.671956 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:47.673699 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d012ec-9b05-4205-949c-28df1bb446bf admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:47.678952 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32119359-a769-4871-9d8e-d3783f99d2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:47.679793 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32119359-a769-4871-9d8e-d3783f99d2aa admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:47.680207 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32119359-a769-4871-9d8e-d3783f99d2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:47.680530 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32119359-a769-4871-9d8e-d3783f99d2aa admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:47.681663 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32119359-a769-4871-9d8e-d3783f99d2aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:47.683733 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32119359-a769-4871-9d8e-d3783f99d2aa admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:47.695642 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1404/2802] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:30:47 2022] POST /identity/v3/projects => generated 363 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:30:47.698214 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1399/2803] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:30:47.699913 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5be5b-7266-41f4-9088-93833c889d3d admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:47.703468 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32119359-a769-4871-9d8e-d3783f99d2aa admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:47.718583 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32119359-a769-4871-9d8e-d3783f99d2aa admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:47.723577 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:47.784514 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5be5b-7266-41f4-9088-93833c889d3d admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:47.784735 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e99bf-c30a-4c0f-9242-ad385c137498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:47.785624 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e99bf-c30a-4c0f-9242-ad385c137498 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:47.785910 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e99bf-c30a-4c0f-9242-ad385c137498 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:47.786188 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e99bf-c30a-4c0f-9242-ad385c137498 admin admin] PATH_INFO: `/v3/projects/20353c2e865a4e9cabccd6e7992b3c0f/users/e9cce82732d945d5b833793b254c4156/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:47.789416 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab838200-0a77-4102-8dce-2924093b5da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:47.790123 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab838200-0a77-4102-8dce-2924093b5da3 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:47.790411 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab838200-0a77-4102-8dce-2924093b5da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:47.790716 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab838200-0a77-4102-8dce-2924093b5da3 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:47.791436 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab838200-0a77-4102-8dce-2924093b5da3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:47.793412 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab838200-0a77-4102-8dce-2924093b5da3 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:47.802651 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e99bf-c30a-4c0f-9242-ad385c137498 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20353c2e865a4e9cabccd6e7992b3c0f, user_id=e9cce82732d945d5b833793b254c4156, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:47.805539 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e99bf-c30a-4c0f-9242-ad385c137498 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:47.815253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1405/2804] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:30:47 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:30:47.819429 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab838200-0a77-4102-8dce-2924093b5da3 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:47.821799 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1400/2805] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:30:47 2022] PUT /identity/v3/projects/20353c2e865a4e9cabccd6e7992b3c0f/users/e9cce82732d945d5b833793b254c4156/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:30:47.829783 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab838200-0a77-4102-8dce-2924093b5da3 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:47.832907 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851886b-53f6-4c02-a4c5-c30bf9581756 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:47.833337 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851886b-53f6-4c02-a4c5-c30bf9581756 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:47.833518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851886b-53f6-4c02-a4c5-c30bf9581756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:47.863334 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-0851886b-53f6-4c02-a4c5-c30bf9581756 None None] MFA Rules not processed for user `e9cce82732d945d5b833793b254c4156`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:47.864632 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-0851886b-53f6-4c02-a4c5-c30bf9581756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:47.917846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a1d9f-a0b6-4e68-beb1-02f5cabcafb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:47.918445 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a1d9f-a0b6-4e68-beb1-02f5cabcafb3 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:47.918731 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a1d9f-a0b6-4e68-beb1-02f5cabcafb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:47.919010 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a1d9f-a0b6-4e68-beb1-02f5cabcafb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:47.920193 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15a1d9f-a0b6-4e68-beb1-02f5cabcafb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:47.923071 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15a1d9f-a0b6-4e68-beb1-02f5cabcafb3 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:47.937306 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1401/2806] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:47 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:47.939852 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1406/2807] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:30:47.944789 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0851886b-53f6-4c02-a4c5-c30bf9581756 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:47.957449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0851886b-53f6-4c02-a4c5-c30bf9581756 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:47.965286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534cd9b-fb76-4ac9-8713-3d7b8ff6725f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:47.966547 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534cd9b-fb76-4ac9-8713-3d7b8ff6725f admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:47.966844 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534cd9b-fb76-4ac9-8713-3d7b8ff6725f admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:47.967164 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534cd9b-fb76-4ac9-8713-3d7b8ff6725f admin admin] PATH_INFO: `/v3/projects/e2f676f81d5e47f59b7f555f0700a315/users/f7e607e9bda1422b85bb7fd445d95fa2/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:47.973298 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a1d9f-a0b6-4e68-beb1-02f5cabcafb3 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:47.975400 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-f15a1d9f-a0b6-4e68-beb1-02f5cabcafb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:47.991270 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7534cd9b-fb76-4ac9-8713-3d7b8ff6725f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2f676f81d5e47f59b7f555f0700a315, user_id=f7e607e9bda1422b85bb7fd445d95fa2, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:47.994917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7534cd9b-fb76-4ac9-8713-3d7b8ff6725f admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:48.012734 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1402/2808] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:30:47 2022] PUT /identity/v3/projects/e2f676f81d5e47f59b7f555f0700a315/users/f7e607e9bda1422b85bb7fd445d95fa2/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:48.017044 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534cd9b-fb76-4ac9-8713-3d7b8ff6725f admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:48.043662 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a1d9f-a0b6-4e68-beb1-02f5cabcafb3 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:48.087994 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:48.090164 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:48.096808 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534cd9b-fb76-4ac9-8713-3d7b8ff6725f admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:48.106088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3cb08b-71bd-479a-8195-59c5c12ecb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:48.106787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3cb08b-71bd-479a-8195-59c5c12ecb52 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:48.107269 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3cb08b-71bd-479a-8195-59c5c12ecb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:48.107555 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3cb08b-71bd-479a-8195-59c5c12ecb52 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:48.109109 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3cb08b-71bd-479a-8195-59c5c12ecb52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:48.110780 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3cb08b-71bd-479a-8195-59c5c12ecb52 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:48.112994 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c40d4c7-f894-406e-b6db-8005ec5b003b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:48.114161 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40d4c7-f894-406e-b6db-8005ec5b003b service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:48.114715 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40d4c7-f894-406e-b6db-8005ec5b003b service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:48.115433 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40d4c7-f894-406e-b6db-8005ec5b003b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:48.123509 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1403/2809] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:30:48.128342 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3cb08b-71bd-479a-8195-59c5c12ecb52 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:48.150405 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3cb08b-71bd-479a-8195-59c5c12ecb52 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:48.155876 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93501826-b36e-4b17-a492-e5283f63b5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:48.157128 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93501826-b36e-4b17-a492-e5283f63b5f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:48.157128 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93501826-b36e-4b17-a492-e5283f63b5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:48.157464 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93501826-b36e-4b17-a492-e5283f63b5f1 admin admin] PATH_INFO: `/v3/projects/e2f676f81d5e47f59b7f555f0700a315/users/f7e607e9bda1422b85bb7fd445d95fa2/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:48.159992 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93501826-b36e-4b17-a492-e5283f63b5f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2f676f81d5e47f59b7f555f0700a315, user_id=f7e607e9bda1422b85bb7fd445d95fa2, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:48.162913 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93501826-b36e-4b17-a492-e5283f63b5f1 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:48.177339 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1404/2810] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:30:48 2022] PUT /identity/v3/projects/e2f676f81d5e47f59b7f555f0700a315/users/f7e607e9bda1422b85bb7fd445d95fa2/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:30:48.187983 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6ee65-a67e-48e7-ac33-c04e95af34e4 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:48.188306 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6ee65-a67e-48e7-ac33-c04e95af34e4 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:48.188560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6ee65-a67e-48e7-ac33-c04e95af34e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:48.202392 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c40d4c7-f894-406e-b6db-8005ec5b003b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:48.203900 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c40d4c7-f894-406e-b6db-8005ec5b003b service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:48.218434 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1407/2811] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:47 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:48.227881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-17f6ee65-a67e-48e7-ac33-c04e95af34e4 None None] MFA Rules not processed for user `f7e607e9bda1422b85bb7fd445d95fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:48.229108 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-17f6ee65-a67e-48e7-ac33-c04e95af34e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:48.239295 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c40d4c7-f894-406e-b6db-8005ec5b003b service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:48.255705 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c40d4c7-f894-406e-b6db-8005ec5b003b service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:48.345611 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab7c58-1cf0-4001-b639-a8d36329a2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:48.346872 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab7c58-1cf0-4001-b639-a8d36329a2c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:48.347392 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab7c58-1cf0-4001-b639-a8d36329a2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:48.347872 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab7c58-1cf0-4001-b639-a8d36329a2c4 admin admin] PATH_INFO: `/v3/users/e9cce82732d945d5b833793b254c4156` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:48.348061 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1405/2812] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:48 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:48.355781 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab7c58-1cf0-4001-b639-a8d36329a2c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9cce82732d945d5b833793b254c4156)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:48.357626 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab7c58-1cf0-4001-b639-a8d36329a2c4 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:48.363515 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6ee65-a67e-48e7-ac33-c04e95af34e4 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:48.367165 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-17f6ee65-a67e-48e7-ac33-c04e95af34e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:48.440685 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-beab7c58-1cf0-4001-b639-a8d36329a2c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9cce82732d945d5b833793b254c4156'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:48.457071 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-beab7c58-1cf0-4001-b639-a8d36329a2c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9cce82732d945d5b833793b254c4156'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:48.457919 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-beab7c58-1cf0-4001-b639-a8d36329a2c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9cce82732d945d5b833793b254c4156'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:48.458611 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-beab7c58-1cf0-4001-b639-a8d36329a2c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9cce82732d945d5b833793b254c4156'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:48.465852 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6ee65-a67e-48e7-ac33-c04e95af34e4 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:48.468001 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-beab7c58-1cf0-4001-b639-a8d36329a2c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9cce82732d945d5b833793b254c4156'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:48.470354 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f68ec-41fa-4196-b4dd-a9ff96d1ae87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:48.471317 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f68ec-41fa-4196-b4dd-a9ff96d1ae87 service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:48.471604 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f68ec-41fa-4196-b4dd-a9ff96d1ae87 service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:48.471883 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f68ec-41fa-4196-b4dd-a9ff96d1ae87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:48.473806 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-cf1f68ec-41fa-4196-b4dd-a9ff96d1ae87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:48.486740 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1408/2813] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:30:48 2022] DELETE /identity/v3/users/e9cce82732d945d5b833793b254c4156 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:30:48.490828 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f68ec-41fa-4196-b4dd-a9ff96d1ae87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:48.491552 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f68ec-41fa-4196-b4dd-a9ff96d1ae87 service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:48.491909 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d1da7-6116-4d46-8198-43452e2ebc52 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:48.492254 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d1da7-6116-4d46-8198-43452e2ebc52 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:48.492534 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d1da7-6116-4d46-8198-43452e2ebc52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:48.502654 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1406/2814] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:48 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:48.513115 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-3a6d1da7-6116-4d46-8198-43452e2ebc52 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:48.514653 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-3a6d1da7-6116-4d46-8198-43452e2ebc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:48.524772 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:48.630243 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1409/2815] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:30:48.643282 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f68ec-41fa-4196-b4dd-a9ff96d1ae87 service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:48.645021 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-cf1f68ec-41fa-4196-b4dd-a9ff96d1ae87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:48.755594 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f68ec-41fa-4196-b4dd-a9ff96d1ae87 service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:48.790407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:48.834152 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1220bad-0956-4fc0-b11b-d32493a4d23b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:48.834827 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1220bad-0956-4fc0-b11b-d32493a4d23b service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:48.835200 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1220bad-0956-4fc0-b11b-d32493a4d23b service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:48.835539 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1220bad-0956-4fc0-b11b-d32493a4d23b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:48.848687 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1220bad-0956-4fc0-b11b-d32493a4d23b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:48.850685 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1220bad-0956-4fc0-b11b-d32493a4d23b service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:48.880034 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1407/2816] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:49.162827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d1da7-6116-4d46-8198-43452e2ebc52 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:49.164839 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-3a6d1da7-6116-4d46-8198-43452e2ebc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:49.179580 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d1da7-6116-4d46-8198-43452e2ebc52 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:49.184580 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10959992-7c2a-46c1-a2c4-2cf65d9d9808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:49.185689 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10959992-7c2a-46c1-a2c4-2cf65d9d9808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:49.185811 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10959992-7c2a-46c1-a2c4-2cf65d9d9808 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:49.186022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10959992-7c2a-46c1-a2c4-2cf65d9d9808 admin admin] PATH_INFO: `/v3/users/9d5a553e2aa2488581ccefcfbaf70ca3` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:49.187598 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10959992-7c2a-46c1-a2c4-2cf65d9d9808 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d5a553e2aa2488581ccefcfbaf70ca3)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:49.189477 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10959992-7c2a-46c1-a2c4-2cf65d9d9808 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:49.245840 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-10959992-7c2a-46c1-a2c4-2cf65d9d9808 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d5a553e2aa2488581ccefcfbaf70ca3'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:49.263751 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-10959992-7c2a-46c1-a2c4-2cf65d9d9808 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d5a553e2aa2488581ccefcfbaf70ca3'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:49.264538 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-10959992-7c2a-46c1-a2c4-2cf65d9d9808 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d5a553e2aa2488581ccefcfbaf70ca3'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:49.265171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-10959992-7c2a-46c1-a2c4-2cf65d9d9808 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d5a553e2aa2488581ccefcfbaf70ca3'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:49.273996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-10959992-7c2a-46c1-a2c4-2cf65d9d9808 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d5a553e2aa2488581ccefcfbaf70ca3'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:49.295444 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1410/2817] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:30:49 2022] DELETE /identity/v3/users/9d5a553e2aa2488581ccefcfbaf70ca3 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:49.297822 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93d6e0-210e-4bc9-a124-fe06cf95c29f None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:49.298860 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93d6e0-210e-4bc9-a124-fe06cf95c29f None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:49.299448 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93d6e0-210e-4bc9-a124-fe06cf95c29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:49.349959 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-4f93d6e0-210e-4bc9-a124-fe06cf95c29f None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:49.353360 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-4f93d6e0-210e-4bc9-a124-fe06cf95c29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:49.454638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1408/2818] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:49.474478 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10959992-7c2a-46c1-a2c4-2cf65d9d9808 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:49.476201 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-10959992-7c2a-46c1-a2c4-2cf65d9d9808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:49.539216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f93d6e0-210e-4bc9-a124-fe06cf95c29f None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:49.541849 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-4f93d6e0-210e-4bc9-a124-fe06cf95c29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:49.563133 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10959992-7c2a-46c1-a2c4-2cf65d9d9808 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:49.578178 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecae9a0-c743-4527-b57c-07ee69c17663 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:49.580131 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecae9a0-c743-4527-b57c-07ee69c17663 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:49.580549 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f93d6e0-210e-4bc9-a124-fe06cf95c29f None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:49.580743 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecae9a0-c743-4527-b57c-07ee69c17663 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:49.581744 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecae9a0-c743-4527-b57c-07ee69c17663 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:49.588241 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f72a9b-a871-4def-8696-63744c850017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:49.589837 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f72a9b-a871-4def-8696-63744c850017 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:49.590348 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f72a9b-a871-4def-8696-63744c850017 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:49.590866 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f72a9b-a871-4def-8696-63744c850017 admin admin] PATH_INFO: `/v3/projects/20353c2e865a4e9cabccd6e7992b3c0f` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:49.593249 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f72a9b-a871-4def-8696-63744c850017 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20353c2e865a4e9cabccd6e7992b3c0f)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:49.597066 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f72a9b-a871-4def-8696-63744c850017 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:49.601725 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecae9a0-c743-4527-b57c-07ee69c17663 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:49.603752 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecae9a0-c743-4527-b57c-07ee69c17663 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:49.618408 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1411/2819] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:49.680449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-c8f72a9b-a871-4def-8696-63744c850017 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20353c2e865a4e9cabccd6e7992b3c0f', 'request_id': 'req-c8f72a9b-a871-4def-8696-63744c850017'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:49.693055 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-c8f72a9b-a871-4def-8696-63744c850017 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20353c2e865a4e9cabccd6e7992b3c0f', 'request_id': 'req-c8f72a9b-a871-4def-8696-63744c850017'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:49.709966 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-c8f72a9b-a871-4def-8696-63744c850017 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20353c2e865a4e9cabccd6e7992b3c0f', 'request_id': 'req-c8f72a9b-a871-4def-8696-63744c850017'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:49.729942 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-c8f72a9b-a871-4def-8696-63744c850017 admin admin] The token cache is being invalidate because project 20353c2e865a4e9cabccd6e7992b3c0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:30:49.730391 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-c8f72a9b-a871-4def-8696-63744c850017 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:49.731418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-c8f72a9b-a871-4def-8696-63744c850017 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:49.734771 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1409/2820] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:30:49 2022] DELETE /identity/v3/projects/20353c2e865a4e9cabccd6e7992b3c0f => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:30:49.741839 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b2a702-87a2-4b2a-9380-670676a9cf03 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:49.742231 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b2a702-87a2-4b2a-9380-670676a9cf03 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:49.742599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b2a702-87a2-4b2a-9380-670676a9cf03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:49.763500 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-a3b2a702-87a2-4b2a-9380-670676a9cf03 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:49.764814 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-a3b2a702-87a2-4b2a-9380-670676a9cf03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:49.790556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:49.851226 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1412/2821] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:30:49.860345 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9dafb-b832-4cd5-ab4f-95d5b1b320e5 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:49.860678 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9dafb-b832-4cd5-ab4f-95d5b1b320e5 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:49.860931 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9dafb-b832-4cd5-ab4f-95d5b1b320e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:49.880288 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-85e9dafb-b832-4cd5-ab4f-95d5b1b320e5 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:49.882037 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-85e9dafb-b832-4cd5-ab4f-95d5b1b320e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:49.891655 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1410/2822] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:49.896025 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b2a702-87a2-4b2a-9380-670676a9cf03 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:49.906722 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b2a702-87a2-4b2a-9380-670676a9cf03 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:49.912247 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3437fc-b00d-44cd-a551-ccfc67398842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:49.912942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3437fc-b00d-44cd-a551-ccfc67398842 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:49.913242 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3437fc-b00d-44cd-a551-ccfc67398842 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:49.913499 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3437fc-b00d-44cd-a551-ccfc67398842 admin admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:49.914270 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3437fc-b00d-44cd-a551-ccfc67398842 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:30:49.914524 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3437fc-b00d-44cd-a551-ccfc67398842 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:49.916692 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3437fc-b00d-44cd-a551-ccfc67398842 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:49.933303 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1413/2823] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 15:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:30:49.938335 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9dafb-b832-4cd5-ab4f-95d5b1b320e5 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:49.948578 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9dafb-b832-4cd5-ab4f-95d5b1b320e5 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:49.954533 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d516ce-5127-43f3-8f96-182598080bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:49.955281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d516ce-5127-43f3-8f96-182598080bd1 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:49.955563 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d516ce-5127-43f3-8f96-182598080bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:49.955908 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d516ce-5127-43f3-8f96-182598080bd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:49.956783 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d516ce-5127-43f3-8f96-182598080bd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:49.958713 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d516ce-5127-43f3-8f96-182598080bd1 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:49.987596 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1411/2824] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:30:49 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:30:49.991958 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3437fc-b00d-44cd-a551-ccfc67398842 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:50.003265 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3437fc-b00d-44cd-a551-ccfc67398842 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:50.083271 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96c4f6-2483-4d38-b2d9-5a00d2b78f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:50.083949 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96c4f6-2483-4d38-b2d9-5a00d2b78f4e admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:50.084966 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96c4f6-2483-4d38-b2d9-5a00d2b78f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:50.085295 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96c4f6-2483-4d38-b2d9-5a00d2b78f4e admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:50.088352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b96c4f6-2483-4d38-b2d9-5a00d2b78f4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:50.090894 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b96c4f6-2483-4d38-b2d9-5a00d2b78f4e admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:50.120087 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1414/2825] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:30:49 2022] POST /identity/v3/users => generated 283 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:30:50.124601 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d516ce-5127-43f3-8f96-182598080bd1 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:50.136301 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d516ce-5127-43f3-8f96-182598080bd1 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:50.141205 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52975950-1494-4a81-b2e0-f7b8040ed865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:50.141900 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52975950-1494-4a81-b2e0-f7b8040ed865 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:50.143167 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52975950-1494-4a81-b2e0-f7b8040ed865 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:50.143167 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52975950-1494-4a81-b2e0-f7b8040ed865 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:50.143331 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52975950-1494-4a81-b2e0-f7b8040ed865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:50.144887 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52975950-1494-4a81-b2e0-f7b8040ed865 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:50.159209 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1412/2826] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:30:50.162074 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96c4f6-2483-4d38-b2d9-5a00d2b78f4e admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:50.174206 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96c4f6-2483-4d38-b2d9-5a00d2b78f4e admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:50.181066 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f993b6d-a44a-42eb-b894-583cb75afd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:50.182182 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f993b6d-a44a-42eb-b894-583cb75afd13 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:50.182290 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f993b6d-a44a-42eb-b894-583cb75afd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:50.182625 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f993b6d-a44a-42eb-b894-583cb75afd13 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:50.184143 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f993b6d-a44a-42eb-b894-583cb75afd13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:50.186837 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f993b6d-a44a-42eb-b894-583cb75afd13 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:50.193489 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52975950-1494-4a81-b2e0-f7b8040ed865 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:50.195588 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-52975950-1494-4a81-b2e0-f7b8040ed865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:50.204942 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1415/2827] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:30:50.206972 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f993b6d-a44a-42eb-b894-583cb75afd13 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:50.216267 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52975950-1494-4a81-b2e0-f7b8040ed865 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:50.223297 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912243d-cc32-4f83-806a-ac8aab2f5a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:50.224199 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912243d-cc32-4f83-806a-ac8aab2f5a70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:50.224491 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912243d-cc32-4f83-806a-ac8aab2f5a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:50.224769 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912243d-cc32-4f83-806a-ac8aab2f5a70 admin admin] PATH_INFO: `/v3/projects/ac0ccd08607644ee89c9c8e23f0ae369` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:50.225239 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f993b6d-a44a-42eb-b894-583cb75afd13 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:50.226037 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1912243d-cc32-4f83-806a-ac8aab2f5a70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac0ccd08607644ee89c9c8e23f0ae369)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:50.228736 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1912243d-cc32-4f83-806a-ac8aab2f5a70 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:50.241974 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb28f92-ff0b-42c5-8c7a-48c4a88a805d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:50.243361 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:50.243475 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb28f92-ff0b-42c5-8c7a-48c4a88a805d admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:50.243475 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb28f92-ff0b-42c5-8c7a-48c4a88a805d admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:50.243717 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb28f92-ff0b-42c5-8c7a-48c4a88a805d admin admin] PATH_INFO: `/v3/projects/16aba7fe10de4afa9c6b33ca31afbc5c/users/3776ccc4048b48dbbf6970975d5f054f/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:50.276089 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb28f92-ff0b-42c5-8c7a-48c4a88a805d admin admin] RBAC: Authorizing `identity:create_grant(project_id=16aba7fe10de4afa9c6b33ca31afbc5c, user_id=3776ccc4048b48dbbf6970975d5f054f, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:50.278796 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb28f92-ff0b-42c5-8c7a-48c4a88a805d admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:50.319748 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1416/2828] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:30:50 2022] PUT /identity/v3/projects/16aba7fe10de4afa9c6b33ca31afbc5c/users/3776ccc4048b48dbbf6970975d5f054f/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:30:50.324355 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7911c90-5da4-4a56-b4f7-e4399b22d99b None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:50.324727 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7911c90-5da4-4a56-b4f7-e4399b22d99b None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:50.324931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7911c90-5da4-4a56-b4f7-e4399b22d99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:50.354376 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-1912243d-cc32-4f83-806a-ac8aab2f5a70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac0ccd08607644ee89c9c8e23f0ae369', 'request_id': 'req-1912243d-cc32-4f83-806a-ac8aab2f5a70'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:50.354991 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-a7911c90-5da4-4a56-b4f7-e4399b22d99b None None] MFA Rules not processed for user `3776ccc4048b48dbbf6970975d5f054f`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:50.356270 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-a7911c90-5da4-4a56-b4f7-e4399b22d99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:50.363951 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-1912243d-cc32-4f83-806a-ac8aab2f5a70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac0ccd08607644ee89c9c8e23f0ae369', 'request_id': 'req-1912243d-cc32-4f83-806a-ac8aab2f5a70'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:50.372105 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-1912243d-cc32-4f83-806a-ac8aab2f5a70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac0ccd08607644ee89c9c8e23f0ae369', 'request_id': 'req-1912243d-cc32-4f83-806a-ac8aab2f5a70'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:50.389259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-1912243d-cc32-4f83-806a-ac8aab2f5a70 admin admin] The token cache is being invalidate because project ac0ccd08607644ee89c9c8e23f0ae369 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:30:50.389587 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-1912243d-cc32-4f83-806a-ac8aab2f5a70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:50.390271 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-1912243d-cc32-4f83-806a-ac8aab2f5a70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:50.392388 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1413/2829] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:30:50 2022] DELETE /identity/v3/projects/ac0ccd08607644ee89c9c8e23f0ae369 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:50.396649 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912243d-cc32-4f83-806a-ac8aab2f5a70 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:50.398043 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-1912243d-cc32-4f83-806a-ac8aab2f5a70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:50.443995 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:50.458401 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1417/2830] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:50 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:30:50.471408 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7911c90-5da4-4a56-b4f7-e4399b22d99b None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:50.475247 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-a7911c90-5da4-4a56-b4f7-e4399b22d99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:50.507694 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912243d-cc32-4f83-806a-ac8aab2f5a70 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:50.514568 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04fe69b-3449-4d08-b969-cf3d67cc8678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:50.515931 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04fe69b-3449-4d08-b969-cf3d67cc8678 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:50.516276 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04fe69b-3449-4d08-b969-cf3d67cc8678 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:50.516616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04fe69b-3449-4d08-b969-cf3d67cc8678 admin admin] PATH_INFO: `/v3/users/f7e607e9bda1422b85bb7fd445d95fa2` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:50.518003 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04fe69b-3449-4d08-b969-cf3d67cc8678 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7e607e9bda1422b85bb7fd445d95fa2)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:50.520334 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04fe69b-3449-4d08-b969-cf3d67cc8678 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:50.557796 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7911c90-5da4-4a56-b4f7-e4399b22d99b None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:50.563687 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b63ce0d-d74a-4c32-b129-2b37f31f3b76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:50.564816 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63ce0d-d74a-4c32-b129-2b37f31f3b76 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:50.565133 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63ce0d-d74a-4c32-b129-2b37f31f3b76 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:50.565420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63ce0d-d74a-4c32-b129-2b37f31f3b76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:50.582304 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b63ce0d-d74a-4c32-b129-2b37f31f3b76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:50.584049 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b63ce0d-d74a-4c32-b129-2b37f31f3b76 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:50.592002 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-f04fe69b-3449-4d08-b969-cf3d67cc8678 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7e607e9bda1422b85bb7fd445d95fa2'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:50.594946 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1418/2831] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:50 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:50.608862 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-f04fe69b-3449-4d08-b969-cf3d67cc8678 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7e607e9bda1422b85bb7fd445d95fa2'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:50.610350 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-f04fe69b-3449-4d08-b969-cf3d67cc8678 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7e607e9bda1422b85bb7fd445d95fa2'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:50.610850 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-f04fe69b-3449-4d08-b969-cf3d67cc8678 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7e607e9bda1422b85bb7fd445d95fa2'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:50.619362 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-f04fe69b-3449-4d08-b969-cf3d67cc8678 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7e607e9bda1422b85bb7fd445d95fa2'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:50.641594 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1414/2832] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:30:50 2022] DELETE /identity/v3/users/f7e607e9bda1422b85bb7fd445d95fa2 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:30:51.054753 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b63ce0d-d74a-4c32-b129-2b37f31f3b76 service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:51.070379 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b63ce0d-d74a-4c32-b129-2b37f31f3b76 service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:51.160282 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84660cf-accf-4f1e-9742-20ce87f85a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:51.161154 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84660cf-accf-4f1e-9742-20ce87f85a3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:51.161435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84660cf-accf-4f1e-9742-20ce87f85a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:51.161709 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84660cf-accf-4f1e-9742-20ce87f85a3d admin admin] PATH_INFO: `/v3/projects/e2f676f81d5e47f59b7f555f0700a315` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:51.163065 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84660cf-accf-4f1e-9742-20ce87f85a3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2f676f81d5e47f59b7f555f0700a315)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:51.165250 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84660cf-accf-4f1e-9742-20ce87f85a3d admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:51.215135 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-d84660cf-accf-4f1e-9742-20ce87f85a3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2f676f81d5e47f59b7f555f0700a315', 'request_id': 'req-d84660cf-accf-4f1e-9742-20ce87f85a3d'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:51.221260 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-d84660cf-accf-4f1e-9742-20ce87f85a3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2f676f81d5e47f59b7f555f0700a315', 'request_id': 'req-d84660cf-accf-4f1e-9742-20ce87f85a3d'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:51.229275 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-d84660cf-accf-4f1e-9742-20ce87f85a3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2f676f81d5e47f59b7f555f0700a315', 'request_id': 'req-d84660cf-accf-4f1e-9742-20ce87f85a3d'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:51.241128 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-d84660cf-accf-4f1e-9742-20ce87f85a3d admin admin] The token cache is being invalidate because project e2f676f81d5e47f59b7f555f0700a315 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:30:51.241420 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-d84660cf-accf-4f1e-9742-20ce87f85a3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:51.242093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-d84660cf-accf-4f1e-9742-20ce87f85a3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:51.243886 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1419/2833] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:30:51 2022] DELETE /identity/v3/projects/e2f676f81d5e47f59b7f555f0700a315 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:51.249266 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8960c-8f0b-47fb-baba-05b04b95e6fd None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:51.249726 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8960c-8f0b-47fb-baba-05b04b95e6fd None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:51.249826 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8960c-8f0b-47fb-baba-05b04b95e6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:51.268957 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-c9a8960c-8f0b-47fb-baba-05b04b95e6fd None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:51.270176 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-c9a8960c-8f0b-47fb-baba-05b04b95e6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:51.275899 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:51.334764 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1415/2834] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:51.342763 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033f442-0854-4287-bf89-da2fff7c3915 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:51.343106 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033f442-0854-4287-bf89-da2fff7c3915 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:51.343390 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033f442-0854-4287-bf89-da2fff7c3915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:51.361921 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-0033f442-0854-4287-bf89-da2fff7c3915 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:51.363171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-0033f442-0854-4287-bf89-da2fff7c3915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:51.371352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1420/2835] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:51.376007 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a8960c-8f0b-47fb-baba-05b04b95e6fd None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:51.388066 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a8960c-8f0b-47fb-baba-05b04b95e6fd None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:51.396126 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3891e7-043e-4259-bf0c-471f033b92ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:51.396829 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3891e7-043e-4259-bf0c-471f033b92ef admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:51.397117 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3891e7-043e-4259-bf0c-471f033b92ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:51.397397 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3891e7-043e-4259-bf0c-471f033b92ef admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:51.398153 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3891e7-043e-4259-bf0c-471f033b92ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:30:51.398465 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3891e7-043e-4259-bf0c-471f033b92ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:51.400154 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3891e7-043e-4259-bf0c-471f033b92ef admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:51.413865 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1416/2836] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 15:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:30:51.418314 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033f442-0854-4287-bf89-da2fff7c3915 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:51.428270 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033f442-0854-4287-bf89-da2fff7c3915 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:51.433673 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d313c-0526-48f4-a5e7-48a1d7e11ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:51.434405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d313c-0526-48f4-a5e7-48a1d7e11ca0 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:51.434405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d313c-0526-48f4-a5e7-48a1d7e11ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:51.434941 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d313c-0526-48f4-a5e7-48a1d7e11ca0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:51.435713 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d313c-0526-48f4-a5e7-48a1d7e11ca0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:51.438174 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d313c-0526-48f4-a5e7-48a1d7e11ca0 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:51.461978 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:51.470038 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1421/2837] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:30:51 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:30:51.474550 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3891e7-043e-4259-bf0c-471f033b92ef admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:51.486853 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3891e7-043e-4259-bf0c-471f033b92ef admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:51.565964 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba0da9-578a-42b4-a176-08bc00f4a101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:51.566691 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba0da9-578a-42b4-a176-08bc00f4a101 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:51.567107 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba0da9-578a-42b4-a176-08bc00f4a101 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:51.567396 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba0da9-578a-42b4-a176-08bc00f4a101 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:51.568109 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cba0da9-578a-42b4-a176-08bc00f4a101 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:51.570144 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cba0da9-578a-42b4-a176-08bc00f4a101 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:51.599535 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1417/2838] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:30:51 2022] POST /identity/v3/users => generated 284 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:30:51.604336 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d313c-0526-48f4-a5e7-48a1d7e11ca0 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:51.615409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d313c-0526-48f4-a5e7-48a1d7e11ca0 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:51.620320 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce30c6-2da3-415c-a27d-52cb1dce2385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:51.621001 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce30c6-2da3-415c-a27d-52cb1dce2385 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:51.621290 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce30c6-2da3-415c-a27d-52cb1dce2385 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:51.621565 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce30c6-2da3-415c-a27d-52cb1dce2385 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:51.622333 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ce30c6-2da3-415c-a27d-52cb1dce2385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:51.624006 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ce30c6-2da3-415c-a27d-52cb1dce2385 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:51.636577 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1422/2839] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:30:51.640751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba0da9-578a-42b4-a176-08bc00f4a101 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:51.654481 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba0da9-578a-42b4-a176-08bc00f4a101 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:51.662369 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a5097-90a1-47ff-bd89-b6c372da0191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:51.663939 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a5097-90a1-47ff-bd89-b6c372da0191 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:51.664243 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a5097-90a1-47ff-bd89-b6c372da0191 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:51.664524 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a5097-90a1-47ff-bd89-b6c372da0191 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:51.667200 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a5097-90a1-47ff-bd89-b6c372da0191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:51.668918 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a5097-90a1-47ff-bd89-b6c372da0191 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:51.683109 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1418/2840] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:30:51.685088 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce30c6-2da3-415c-a27d-52cb1dce2385 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:51.698335 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce30c6-2da3-415c-a27d-52cb1dce2385 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:51.707504 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f93d1-dcad-4e16-9b8c-da282430d67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:51.708618 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f93d1-dcad-4e16-9b8c-da282430d67e admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:51.708981 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f93d1-dcad-4e16-9b8c-da282430d67e admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:51.709347 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f93d1-dcad-4e16-9b8c-da282430d67e admin admin] PATH_INFO: `/v3/projects/d3f64105e005436eb1d170e7dcf808aa/users/622fed09b9c34874a93ce052b57b6bce/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:51.731824 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f93d1-dcad-4e16-9b8c-da282430d67e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3f64105e005436eb1d170e7dcf808aa, user_id=622fed09b9c34874a93ce052b57b6bce, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:51.736105 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f93d1-dcad-4e16-9b8c-da282430d67e admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:51.755292 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1423/2841] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:30:51 2022] PUT /identity/v3/projects/d3f64105e005436eb1d170e7dcf808aa/users/622fed09b9c34874a93ce052b57b6bce/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:30:51.767878 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3b767-9ca1-4d8b-bc84-a3d75e101dbf None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:51.768169 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3b767-9ca1-4d8b-bc84-a3d75e101dbf None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:51.768449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3b767-9ca1-4d8b-bc84-a3d75e101dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:51.798835 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-29d3b767-9ca1-4d8b-bc84-a3d75e101dbf None None] MFA Rules not processed for user `622fed09b9c34874a93ce052b57b6bce`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:51.800129 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-29d3b767-9ca1-4d8b-bc84-a3d75e101dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:51.897340 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1419/2842] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:51 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:51.902218 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f93d1-dcad-4e16-9b8c-da282430d67e admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:51.983272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f93d1-dcad-4e16-9b8c-da282430d67e admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:51.988200 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac6db1-2162-43b6-9c0c-014ede780c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:51.988896 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac6db1-2162-43b6-9c0c-014ede780c21 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:51.989182 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac6db1-2162-43b6-9c0c-014ede780c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:51.989462 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac6db1-2162-43b6-9c0c-014ede780c21 admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:51.990317 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac6db1-2162-43b6-9c0c-014ede780c21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:51.992311 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac6db1-2162-43b6-9c0c-014ede780c21 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:52.023525 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1424/2843] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:30:51 2022] POST /identity/v3/projects => generated 369 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:30:52.027751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3b767-9ca1-4d8b-bc84-a3d75e101dbf None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:52.038693 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3b767-9ca1-4d8b-bc84-a3d75e101dbf None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:52.042557 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:52.115234 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763cead4-bf93-491e-a71e-85ecb9a6847e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:52.115234 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763cead4-bf93-491e-a71e-85ecb9a6847e admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:52.115234 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763cead4-bf93-491e-a71e-85ecb9a6847e admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:52.115234 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763cead4-bf93-491e-a71e-85ecb9a6847e admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:52.115741 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763cead4-bf93-491e-a71e-85ecb9a6847e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:52.115741 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763cead4-bf93-491e-a71e-85ecb9a6847e admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:52.133836 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1420/2844] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:30:52 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:30:52.137781 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac6db1-2162-43b6-9c0c-014ede780c21 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:52.149327 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac6db1-2162-43b6-9c0c-014ede780c21 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:52.156515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d3758-bfbd-4dbf-8e28-d74b6d6531b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:52.157203 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d3758-bfbd-4dbf-8e28-d74b6d6531b9 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:52.157491 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d3758-bfbd-4dbf-8e28-d74b6d6531b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:52.157771 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d3758-bfbd-4dbf-8e28-d74b6d6531b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:52.158684 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129d3758-bfbd-4dbf-8e28-d74b6d6531b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:52.160344 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129d3758-bfbd-4dbf-8e28-d74b6d6531b9 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:52.173971 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1425/2845] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:30:52.180207 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763cead4-bf93-491e-a71e-85ecb9a6847e admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:52.197569 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763cead4-bf93-491e-a71e-85ecb9a6847e admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:52.204293 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090adfc-f2cc-4b78-ae1b-da35db9490b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:52.205227 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090adfc-f2cc-4b78-ae1b-da35db9490b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:52.205508 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090adfc-f2cc-4b78-ae1b-da35db9490b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:52.205786 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090adfc-f2cc-4b78-ae1b-da35db9490b1 admin admin] PATH_INFO: `/v3/projects/8a89c3131d5d45efb1f0c33535f15b54/users/280f9d7d272f4ac7bc29719898507724/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:52.226180 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090adfc-f2cc-4b78-ae1b-da35db9490b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a89c3131d5d45efb1f0c33535f15b54, user_id=280f9d7d272f4ac7bc29719898507724, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:52.229558 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090adfc-f2cc-4b78-ae1b-da35db9490b1 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:52.247168 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1421/2846] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:30:52 2022] PUT /identity/v3/projects/8a89c3131d5d45efb1f0c33535f15b54/users/280f9d7d272f4ac7bc29719898507724/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:52.250795 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d3758-bfbd-4dbf-8e28-d74b6d6531b9 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:52.262759 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d3758-bfbd-4dbf-8e28-d74b6d6531b9 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:52.322894 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d70f24-292d-4d83-a834-a480845cc7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:52.323594 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d70f24-292d-4d83-a834-a480845cc7aa admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:52.323874 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d70f24-292d-4d83-a834-a480845cc7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:52.324146 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d70f24-292d-4d83-a834-a480845cc7aa admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:52.324883 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d70f24-292d-4d83-a834-a480845cc7aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:52.326439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d70f24-292d-4d83-a834-a480845cc7aa admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:52.336682 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1426/2847] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:30:52.340921 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090adfc-f2cc-4b78-ae1b-da35db9490b1 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:52.353125 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090adfc-f2cc-4b78-ae1b-da35db9490b1 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:52.358619 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd7b26-4d89-47f9-a970-71a6a151b11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:52.359573 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd7b26-4d89-47f9-a970-71a6a151b11e admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:52.359860 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd7b26-4d89-47f9-a970-71a6a151b11e admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:52.360142 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd7b26-4d89-47f9-a970-71a6a151b11e admin admin] PATH_INFO: `/v3/projects/8a89c3131d5d45efb1f0c33535f15b54/users/280f9d7d272f4ac7bc29719898507724/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:52.362711 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd7b26-4d89-47f9-a970-71a6a151b11e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a89c3131d5d45efb1f0c33535f15b54, user_id=280f9d7d272f4ac7bc29719898507724, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:52.366371 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd7b26-4d89-47f9-a970-71a6a151b11e admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:52.377744 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1422/2848] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:30:52 2022] PUT /identity/v3/projects/8a89c3131d5d45efb1f0c33535f15b54/users/280f9d7d272f4ac7bc29719898507724/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:52.388109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79a1bb-f5e7-4b2d-bf85-9940eb0c2e11 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:52.388474 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79a1bb-f5e7-4b2d-bf85-9940eb0c2e11 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:52.388827 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79a1bb-f5e7-4b2d-bf85-9940eb0c2e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:52.416779 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-3a79a1bb-f5e7-4b2d-bf85-9940eb0c2e11 None None] MFA Rules not processed for user `280f9d7d272f4ac7bc29719898507724`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:52.417965 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-3a79a1bb-f5e7-4b2d-bf85-9940eb0c2e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:52.442176 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:52.490633 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1427/2849] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:52 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:52.502345 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd7b26-4d89-47f9-a970-71a6a151b11e admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:52.504107 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-97fd7b26-4d89-47f9-a970-71a6a151b11e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:52.574112 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd7b26-4d89-47f9-a970-71a6a151b11e admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:52.578254 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc973b67-d483-4c49-a27b-e6318b75ee27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:52.579994 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc973b67-d483-4c49-a27b-e6318b75ee27 service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:52.580194 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc973b67-d483-4c49-a27b-e6318b75ee27 service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:52.580508 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc973b67-d483-4c49-a27b-e6318b75ee27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:52.592215 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc973b67-d483-4c49-a27b-e6318b75ee27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:52.593650 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc973b67-d483-4c49-a27b-e6318b75ee27 service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:52.604541 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1423/2850] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:52 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:52.651394 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79a1bb-f5e7-4b2d-bf85-9940eb0c2e11 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:52.653592 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-3a79a1bb-f5e7-4b2d-bf85-9940eb0c2e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:52.669061 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79a1bb-f5e7-4b2d-bf85-9940eb0c2e11 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:52.673302 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4955d-044e-4028-be54-9499ce039299 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:52.673976 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4955d-044e-4028-be54-9499ce039299 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:52.674284 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4955d-044e-4028-be54-9499ce039299 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:52.674560 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4955d-044e-4028-be54-9499ce039299 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:52.747557 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf4955d-044e-4028-be54-9499ce039299 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:52.750267 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf4955d-044e-4028-be54-9499ce039299 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:52.766991 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1428/2851] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:52 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:56.409538 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc973b67-d483-4c49-a27b-e6318b75ee27 service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:56.483011 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc973b67-d483-4c49-a27b-e6318b75ee27 service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:56.489252 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c1844-01b4-4941-bbe3-26b1f964ac7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:56.490176 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c1844-01b4-4941-bbe3-26b1f964ac7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:56.490456 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c1844-01b4-4941-bbe3-26b1f964ac7d admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:56.490732 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c1844-01b4-4941-bbe3-26b1f964ac7d admin admin] PATH_INFO: `/v3/users/622fed09b9c34874a93ce052b57b6bce` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:56.491967 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63c1844-01b4-4941-bbe3-26b1f964ac7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=622fed09b9c34874a93ce052b57b6bce)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:56.493950 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63c1844-01b4-4941-bbe3-26b1f964ac7d admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:56.496711 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:56.542720 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-a63c1844-01b4-4941-bbe3-26b1f964ac7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '622fed09b9c34874a93ce052b57b6bce'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:56.554246 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-a63c1844-01b4-4941-bbe3-26b1f964ac7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '622fed09b9c34874a93ce052b57b6bce'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:56.554972 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-a63c1844-01b4-4941-bbe3-26b1f964ac7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '622fed09b9c34874a93ce052b57b6bce'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:56.555687 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-a63c1844-01b4-4941-bbe3-26b1f964ac7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '622fed09b9c34874a93ce052b57b6bce'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:56.562487 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-a63c1844-01b4-4941-bbe3-26b1f964ac7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '622fed09b9c34874a93ce052b57b6bce'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:56.577128 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1424/2852] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:30:56 2022] DELETE /identity/v3/users/622fed09b9c34874a93ce052b57b6bce => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:56.582237 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c9d09-747e-4e37-9ee0-efe2ef20918c None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:56.582755 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c9d09-747e-4e37-9ee0-efe2ef20918c None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:56.582846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c9d09-747e-4e37-9ee0-efe2ef20918c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:56.600300 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-5b9c9d09-747e-4e37-9ee0-efe2ef20918c None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:56.601534 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-5b9c9d09-747e-4e37-9ee0-efe2ef20918c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:56.663402 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1429/2853] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:56.673309 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c1844-01b4-4941-bbe3-26b1f964ac7d admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:56.674825 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-a63c1844-01b4-4941-bbe3-26b1f964ac7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:56.744310 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c1844-01b4-4941-bbe3-26b1f964ac7d admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:56.748314 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e69ba3a-ace1-411c-8187-b70bc1a23751 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:56.748978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69ba3a-ace1-411c-8187-b70bc1a23751 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:56.749259 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69ba3a-ace1-411c-8187-b70bc1a23751 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:56.749533 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69ba3a-ace1-411c-8187-b70bc1a23751 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:56.761350 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e69ba3a-ace1-411c-8187-b70bc1a23751 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:56.764097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e69ba3a-ace1-411c-8187-b70bc1a23751 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:56.779017 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1425/2854] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:57.064396 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c9d09-747e-4e37-9ee0-efe2ef20918c None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:57.065992 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-5b9c9d09-747e-4e37-9ee0-efe2ef20918c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:57.078907 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c9d09-747e-4e37-9ee0-efe2ef20918c None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:57.084417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6374ac-f1d5-4ddc-8a86-4f4557aab24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:57.085363 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6374ac-f1d5-4ddc-8a86-4f4557aab24f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:57.085650 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6374ac-f1d5-4ddc-8a86-4f4557aab24f admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:57.085924 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6374ac-f1d5-4ddc-8a86-4f4557aab24f admin admin] PATH_INFO: `/v3/projects/d3f64105e005436eb1d170e7dcf808aa` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:57.087280 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6374ac-f1d5-4ddc-8a86-4f4557aab24f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3f64105e005436eb1d170e7dcf808aa)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:57.089333 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6374ac-f1d5-4ddc-8a86-4f4557aab24f admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:57.142821 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fb6374ac-f1d5-4ddc-8a86-4f4557aab24f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3f64105e005436eb1d170e7dcf808aa', 'request_id': 'req-fb6374ac-f1d5-4ddc-8a86-4f4557aab24f'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:57.147239 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:57.150387 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fb6374ac-f1d5-4ddc-8a86-4f4557aab24f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3f64105e005436eb1d170e7dcf808aa', 'request_id': 'req-fb6374ac-f1d5-4ddc-8a86-4f4557aab24f'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:57.160417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fb6374ac-f1d5-4ddc-8a86-4f4557aab24f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3f64105e005436eb1d170e7dcf808aa', 'request_id': 'req-fb6374ac-f1d5-4ddc-8a86-4f4557aab24f'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:57.173734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fb6374ac-f1d5-4ddc-8a86-4f4557aab24f admin admin] The token cache is being invalidate because project d3f64105e005436eb1d170e7dcf808aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:30:57.174031 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fb6374ac-f1d5-4ddc-8a86-4f4557aab24f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:57.174861 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-fb6374ac-f1d5-4ddc-8a86-4f4557aab24f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:57.176506 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1430/2855] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:30:57 2022] DELETE /identity/v3/projects/d3f64105e005436eb1d170e7dcf808aa => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:57.180744 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e69ba3a-ace1-411c-8187-b70bc1a23751 service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:57.191027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e69ba3a-ace1-411c-8187-b70bc1a23751 service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:57.250681 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f039423b-5daf-4acb-946c-13a7de470e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:57.251594 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039423b-5daf-4acb-946c-13a7de470e43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:57.251903 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039423b-5daf-4acb-946c-13a7de470e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:57.252180 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039423b-5daf-4acb-946c-13a7de470e43 admin admin] PATH_INFO: `/v3/users/280f9d7d272f4ac7bc29719898507724` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:57.253389 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f039423b-5daf-4acb-946c-13a7de470e43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=280f9d7d272f4ac7bc29719898507724)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:57.255328 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f039423b-5daf-4acb-946c-13a7de470e43 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:57.286474 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:57.320758 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-f039423b-5daf-4acb-946c-13a7de470e43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '280f9d7d272f4ac7bc29719898507724'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:57.338107 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-f039423b-5daf-4acb-946c-13a7de470e43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '280f9d7d272f4ac7bc29719898507724'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:57.339592 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-f039423b-5daf-4acb-946c-13a7de470e43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '280f9d7d272f4ac7bc29719898507724'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:57.339592 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-f039423b-5daf-4acb-946c-13a7de470e43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '280f9d7d272f4ac7bc29719898507724'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:57.349459 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-f039423b-5daf-4acb-946c-13a7de470e43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '280f9d7d272f4ac7bc29719898507724'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:57.371452 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1426/2856] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:30:57 2022] DELETE /identity/v3/users/280f9d7d272f4ac7bc29719898507724 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:57.705074 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6374ac-f1d5-4ddc-8a86-4f4557aab24f admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:57.706620 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-fb6374ac-f1d5-4ddc-8a86-4f4557aab24f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:57.749952 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6374ac-f1d5-4ddc-8a86-4f4557aab24f admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:57.848311 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4949cf4-0dbe-445c-bd46-038dd72e1a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:57.849070 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4949cf4-0dbe-445c-bd46-038dd72e1a1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:57.849352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4949cf4-0dbe-445c-bd46-038dd72e1a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:57.849627 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4949cf4-0dbe-445c-bd46-038dd72e1a1d admin admin] PATH_INFO: `/v3/projects/8a89c3131d5d45efb1f0c33535f15b54` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:57.851005 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4949cf4-0dbe-445c-bd46-038dd72e1a1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a89c3131d5d45efb1f0c33535f15b54)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:57.853601 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4949cf4-0dbe-445c-bd46-038dd72e1a1d admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:57.923414 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-d4949cf4-0dbe-445c-bd46-038dd72e1a1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a89c3131d5d45efb1f0c33535f15b54', 'request_id': 'req-d4949cf4-0dbe-445c-bd46-038dd72e1a1d'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:57.934613 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-d4949cf4-0dbe-445c-bd46-038dd72e1a1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a89c3131d5d45efb1f0c33535f15b54', 'request_id': 'req-d4949cf4-0dbe-445c-bd46-038dd72e1a1d'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:57.946349 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-d4949cf4-0dbe-445c-bd46-038dd72e1a1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a89c3131d5d45efb1f0c33535f15b54', 'request_id': 'req-d4949cf4-0dbe-445c-bd46-038dd72e1a1d'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:57.960782 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-d4949cf4-0dbe-445c-bd46-038dd72e1a1d admin admin] The token cache is being invalidate because project 8a89c3131d5d45efb1f0c33535f15b54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:30:57.961223 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-d4949cf4-0dbe-445c-bd46-038dd72e1a1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:57.962206 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-d4949cf4-0dbe-445c-bd46-038dd72e1a1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:57.964002 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1431/2857] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:30:57 2022] DELETE /identity/v3/projects/8a89c3131d5d45efb1f0c33535f15b54 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:57.969101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f132a1-bc9d-4819-b2d5-56ab0b54e52a None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:57.969632 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f132a1-bc9d-4819-b2d5-56ab0b54e52a None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:57.969718 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f132a1-bc9d-4819-b2d5-56ab0b54e52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:57.989458 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-c6f132a1-bc9d-4819-b2d5-56ab0b54e52a None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:57.991533 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-c6f132a1-bc9d-4819-b2d5-56ab0b54e52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:58.059941 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1427/2858] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:58.069136 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa1763-e5bf-4522-9da9-4bb9d41dd69f None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:58.069791 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa1763-e5bf-4522-9da9-4bb9d41dd69f None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:58.070353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa1763-e5bf-4522-9da9-4bb9d41dd69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:58.094244 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-67aa1763-e5bf-4522-9da9-4bb9d41dd69f None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:58.095814 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-67aa1763-e5bf-4522-9da9-4bb9d41dd69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:58.105604 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1432/2859] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:58.110227 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f132a1-bc9d-4819-b2d5-56ab0b54e52a None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:58.128090 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f132a1-bc9d-4819-b2d5-56ab0b54e52a None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:58.136048 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d02f3-8337-4ae3-a8f1-ea82e0bbddba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:58.137281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d02f3-8337-4ae3-a8f1-ea82e0bbddba admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:58.137852 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d02f3-8337-4ae3-a8f1-ea82e0bbddba admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:58.138355 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d02f3-8337-4ae3-a8f1-ea82e0bbddba admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:58.139694 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d02f3-8337-4ae3-a8f1-ea82e0bbddba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:30:58.140209 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d02f3-8337-4ae3-a8f1-ea82e0bbddba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:58.143008 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d02f3-8337-4ae3-a8f1-ea82e0bbddba admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:58.161359 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1428/2860] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 15:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:30:58.162749 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa1763-e5bf-4522-9da9-4bb9d41dd69f None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:58.172747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa1763-e5bf-4522-9da9-4bb9d41dd69f None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:58.177749 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b5152-7ae6-40a4-8729-8f9092376acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:58.178448 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b5152-7ae6-40a4-8729-8f9092376acf admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:58.178733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b5152-7ae6-40a4-8729-8f9092376acf admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:58.179013 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b5152-7ae6-40a4-8729-8f9092376acf admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:58.179791 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b5152-7ae6-40a4-8729-8f9092376acf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:58.181837 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b5152-7ae6-40a4-8729-8f9092376acf admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:58.211659 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1433/2861] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:30:58 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:30:58.216690 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d02f3-8337-4ae3-a8f1-ea82e0bbddba admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:58.234411 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d02f3-8337-4ae3-a8f1-ea82e0bbddba admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:58.306383 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae93e3dc-a2b9-4c8c-b219-0aec6b7152b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:58.307108 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93e3dc-a2b9-4c8c-b219-0aec6b7152b6 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:58.307398 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93e3dc-a2b9-4c8c-b219-0aec6b7152b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:58.307679 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93e3dc-a2b9-4c8c-b219-0aec6b7152b6 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:58.308389 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae93e3dc-a2b9-4c8c-b219-0aec6b7152b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:58.310331 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae93e3dc-a2b9-4c8c-b219-0aec6b7152b6 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:58.334239 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1429/2862] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:30:58 2022] POST /identity/v3/users => generated 291 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:30:58.338881 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b5152-7ae6-40a4-8729-8f9092376acf admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:58.350253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b5152-7ae6-40a4-8729-8f9092376acf admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:58.355902 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4d7a6-98b0-4b26-a090-56ca2d9449aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:58.356573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4d7a6-98b0-4b26-a090-56ca2d9449aa admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:58.357508 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4d7a6-98b0-4b26-a090-56ca2d9449aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:58.357815 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4d7a6-98b0-4b26-a090-56ca2d9449aa admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:58.358769 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba4d7a6-98b0-4b26-a090-56ca2d9449aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:58.360465 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba4d7a6-98b0-4b26-a090-56ca2d9449aa admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:58.371984 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1434/2863] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:30:58.376407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae93e3dc-a2b9-4c8c-b219-0aec6b7152b6 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:58.387082 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae93e3dc-a2b9-4c8c-b219-0aec6b7152b6 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:58.391958 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d56476-322f-496d-a7c9-5acfa92c3606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:58.392628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d56476-322f-496d-a7c9-5acfa92c3606 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:58.392917 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d56476-322f-496d-a7c9-5acfa92c3606 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:58.393197 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d56476-322f-496d-a7c9-5acfa92c3606 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:58.393931 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d56476-322f-496d-a7c9-5acfa92c3606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:58.395605 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d56476-322f-496d-a7c9-5acfa92c3606 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:58.408358 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1430/2864] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:30:58.412199 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4d7a6-98b0-4b26-a090-56ca2d9449aa admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:58.426562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4d7a6-98b0-4b26-a090-56ca2d9449aa admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:58.432082 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17732520-aac7-4e08-8120-d74532884f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:58.433086 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17732520-aac7-4e08-8120-d74532884f79 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:58.433363 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17732520-aac7-4e08-8120-d74532884f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:58.434168 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17732520-aac7-4e08-8120-d74532884f79 admin admin] PATH_INFO: `/v3/projects/0d4e79824ed849e990b0d1540a1de8a3/users/7897b4f4a4f842fc8a4677661bbb0315/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:58.453375 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17732520-aac7-4e08-8120-d74532884f79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d4e79824ed849e990b0d1540a1de8a3, user_id=7897b4f4a4f842fc8a4677661bbb0315, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:58.456136 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17732520-aac7-4e08-8120-d74532884f79 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:58.470994 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1435/2865] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:30:58 2022] PUT /identity/v3/projects/0d4e79824ed849e990b0d1540a1de8a3/users/7897b4f4a4f842fc8a4677661bbb0315/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:30:58.485468 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520ea06-d56a-457b-a84a-27e15a58cbaa None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:58.485750 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520ea06-d56a-457b-a84a-27e15a58cbaa None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:58.485965 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520ea06-d56a-457b-a84a-27e15a58cbaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:58.505228 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:58.516638 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-d520ea06-d56a-457b-a84a-27e15a58cbaa None None] MFA Rules not processed for user `7897b4f4a4f842fc8a4677661bbb0315`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:58.518572 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-d520ea06-d56a-457b-a84a-27e15a58cbaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:58.589309 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1431/2866] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:58.594388 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17732520-aac7-4e08-8120-d74532884f79 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:58.601337 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:58.665867 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17732520-aac7-4e08-8120-d74532884f79 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:58.677273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb6793a-f948-4bff-bf8a-2b8eae317377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:58.677990 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6793a-f948-4bff-bf8a-2b8eae317377 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:58.678235 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6793a-f948-4bff-bf8a-2b8eae317377 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:58.678538 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6793a-f948-4bff-bf8a-2b8eae317377 admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:58.679352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb6793a-f948-4bff-bf8a-2b8eae317377 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:58.681180 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb6793a-f948-4bff-bf8a-2b8eae317377 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:58.685479 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b796d65-917b-4092-8019-e05567662e9b None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:58.685790 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b796d65-917b-4092-8019-e05567662e9b None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:58.686062 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b796d65-917b-4092-8019-e05567662e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:58.706865 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-0b796d65-917b-4092-8019-e05567662e9b None None] MFA Rules not processed for user `2f8cf6d908314281a3ddc9d9513d39c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:58.708037 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-0b796d65-917b-4092-8019-e05567662e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:58.712038 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1436/2867] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:30:58 2022] POST /identity/v3/projects => generated 381 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:30:58.713159 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb6793a-f948-4bff-bf8a-2b8eae317377 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:58.780641 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1432/2868] 10.222.0.15 () {58 vars in 967 bytes} [Wed Jul 20 15:30:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:58.782366 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb6793a-f948-4bff-bf8a-2b8eae317377 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:58.787884 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55225e9a-ab05-44d9-be68-68ac6efeb2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:58.788800 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55225e9a-ab05-44d9-be68-68ac6efeb2f5 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:58.788988 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55225e9a-ab05-44d9-be68-68ac6efeb2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:58.789210 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55225e9a-ab05-44d9-be68-68ac6efeb2f5 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:58.790037 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55225e9a-ab05-44d9-be68-68ac6efeb2f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:58.792092 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55225e9a-ab05-44d9-be68-68ac6efeb2f5 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:58.829935 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1437/2869] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:30:58 2022] POST /identity/v3/users => generated 297 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:30:58.830236 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b796d65-917b-4092-8019-e05567662e9b None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:58.830963 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55225e9a-ab05-44d9-be68-68ac6efeb2f5 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:58.831884 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-0b796d65-917b-4092-8019-e05567662e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:58.849563 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55225e9a-ab05-44d9-be68-68ac6efeb2f5 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:58.849563 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c3df3-ea37-4d6b-a2d9-1ac1c4da5059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:58.849563 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c3df3-ea37-4d6b-a2d9-1ac1c4da5059 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:58.849563 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c3df3-ea37-4d6b-a2d9-1ac1c4da5059 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:58.849864 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c3df3-ea37-4d6b-a2d9-1ac1c4da5059 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:58.850008 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6c3df3-ea37-4d6b-a2d9-1ac1c4da5059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:58.851687 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6c3df3-ea37-4d6b-a2d9-1ac1c4da5059 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:58.862824 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1438/2870] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:30:58.867175 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c3df3-ea37-4d6b-a2d9-1ac1c4da5059 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:58.877316 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c3df3-ea37-4d6b-a2d9-1ac1c4da5059 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:58.883105 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e15a3-122a-4855-887b-063e0a74d1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:58.884040 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e15a3-122a-4855-887b-063e0a74d1f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:58.884334 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e15a3-122a-4855-887b-063e0a74d1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:58.884727 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e15a3-122a-4855-887b-063e0a74d1f1 admin admin] PATH_INFO: `/v3/projects/565042976fd142dea6924005fbc0ecc5/users/9285c91e885a422381e51d253ae4b879/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:58.899405 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e15a3-122a-4855-887b-063e0a74d1f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=565042976fd142dea6924005fbc0ecc5, user_id=9285c91e885a422381e51d253ae4b879, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:58.902282 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e15a3-122a-4855-887b-063e0a74d1f1 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:58.907293 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b796d65-917b-4092-8019-e05567662e9b None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:58.913527 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd19c5a-fc74-4a8c-b327-dd632554456c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f4844988474d7a91133d77906611a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:58.914197 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd19c5a-fc74-4a8c-b327-dd632554456c service swift] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:58.914495 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd19c5a-fc74-4a8c-b327-dd632554456c service swift] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:58.914828 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd19c5a-fc74-4a8c-b327-dd632554456c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:58.915527 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1439/2871] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:30:58 2022] PUT /identity/v3/projects/565042976fd142dea6924005fbc0ecc5/users/9285c91e885a422381e51d253ae4b879/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:30:58.919479 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e15a3-122a-4855-887b-063e0a74d1f1 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:58.923852 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd19c5a-fc74-4a8c-b327-dd632554456c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:58.925440 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd19c5a-fc74-4a8c-b327-dd632554456c service swift] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:58.933871 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1433/2872] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jul 20 15:30:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:30:58.968687 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:58.986877 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e15a3-122a-4855-887b-063e0a74d1f1 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:58.992096 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a742050-d7aa-446c-899b-59969a0fcbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:58.992812 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a742050-d7aa-446c-899b-59969a0fcbb7 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:58.993192 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a742050-d7aa-446c-899b-59969a0fcbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:58.993498 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a742050-d7aa-446c-899b-59969a0fcbb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:58.994241 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a742050-d7aa-446c-899b-59969a0fcbb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:58.996267 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a742050-d7aa-446c-899b-59969a0fcbb7 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:59.007318 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1440/2873] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:30:59.011670 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd19c5a-fc74-4a8c-b327-dd632554456c service swift] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:59.023219 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd19c5a-fc74-4a8c-b327-dd632554456c service swift] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:59.028926 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945bd480-1c5b-4dd7-a36d-a7f695581004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:59.029840 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bd480-1c5b-4dd7-a36d-a7f695581004 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:59.030123 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bd480-1c5b-4dd7-a36d-a7f695581004 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:59.030462 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bd480-1c5b-4dd7-a36d-a7f695581004 admin admin] PATH_INFO: `/v3/projects/565042976fd142dea6924005fbc0ecc5/users/9285c91e885a422381e51d253ae4b879/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:59.032642 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945bd480-1c5b-4dd7-a36d-a7f695581004 admin admin] RBAC: Authorizing `identity:create_grant(project_id=565042976fd142dea6924005fbc0ecc5, user_id=9285c91e885a422381e51d253ae4b879, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:59.035359 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945bd480-1c5b-4dd7-a36d-a7f695581004 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:59.047395 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1434/2874] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:30:59 2022] PUT /identity/v3/projects/565042976fd142dea6924005fbc0ecc5/users/9285c91e885a422381e51d253ae4b879/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:30:59.057885 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4139f-cead-4de9-aaf2-6b37b9ca54a7 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:59.058174 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4139f-cead-4de9-aaf2-6b37b9ca54a7 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:59.058458 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4139f-cead-4de9-aaf2-6b37b9ca54a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:59.089963 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-f6b4139f-cead-4de9-aaf2-6b37b9ca54a7 None None] MFA Rules not processed for user `9285c91e885a422381e51d253ae4b879`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:59.091410 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-f6b4139f-cead-4de9-aaf2-6b37b9ca54a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:59.177515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1441/2875] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:30:59 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:59.190184 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945bd480-1c5b-4dd7-a36d-a7f695581004 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:59.192027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-945bd480-1c5b-4dd7-a36d-a7f695581004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:59.206865 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:59.252722 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945bd480-1c5b-4dd7-a36d-a7f695581004 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:59.256525 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a0766-1ff2-4b5a-ab7d-1682b66c45f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:59.257183 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a0766-1ff2-4b5a-ab7d-1682b66c45f7 service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:59.257457 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a0766-1ff2-4b5a-ab7d-1682b66c45f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:59.257767 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a0766-1ff2-4b5a-ab7d-1682b66c45f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:59.268453 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8a0766-1ff2-4b5a-ab7d-1682b66c45f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:59.269987 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8a0766-1ff2-4b5a-ab7d-1682b66c45f7 service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:59.285299 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1435/2876] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:59 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:59.611756 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b4139f-cead-4de9-aaf2-6b37b9ca54a7 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:59.621176 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b4139f-cead-4de9-aaf2-6b37b9ca54a7 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:59.625749 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0738241-d876-46d2-acf3-f0b96ee69e4f tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9285c91e885a422381e51d253ae4b879', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '565042976fd142dea6924005fbc0ecc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:59.626450 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0738241-d876-46d2-acf3-f0b96ee69e4f tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:59.626740 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0738241-d876-46d2-acf3-f0b96ee69e4f tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:59.627022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0738241-d876-46d2-acf3-f0b96ee69e4f tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:59.627784 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0738241-d876-46d2-acf3-f0b96ee69e4f tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:30:59.628093 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0738241-d876-46d2-acf3-f0b96ee69e4f tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:59.629693 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0738241-d876-46d2-acf3-f0b96ee69e4f tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:59.641775 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1442/2877] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 15:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:30:59.646024 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a0766-1ff2-4b5a-ab7d-1682b66c45f7 service cinder] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:59.655530 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a0766-1ff2-4b5a-ab7d-1682b66c45f7 service cinder] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:59.661409 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf328443-676f-40a5-974e-94cf3d8bf7aa tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9285c91e885a422381e51d253ae4b879', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '565042976fd142dea6924005fbc0ecc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:59.662075 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf328443-676f-40a5-974e-94cf3d8bf7aa tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:59.662359 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf328443-676f-40a5-974e-94cf3d8bf7aa tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:59.662635 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf328443-676f-40a5-974e-94cf3d8bf7aa tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:59.663351 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf328443-676f-40a5-974e-94cf3d8bf7aa tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:59.665334 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf328443-676f-40a5-974e-94cf3d8bf7aa tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:59.694751 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1436/2878] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:30:59 2022] POST /identity/v3/projects => generated 352 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:30:59.699472 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0738241-d876-46d2-acf3-f0b96ee69e4f tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:59.709375 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0738241-d876-46d2-acf3-f0b96ee69e4f tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:59.774433 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938c114-c6b3-460b-a002-875a67b7bb34 tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9285c91e885a422381e51d253ae4b879', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '565042976fd142dea6924005fbc0ecc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:59.775150 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938c114-c6b3-460b-a002-875a67b7bb34 tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:59.775435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938c114-c6b3-460b-a002-875a67b7bb34 tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:59.775717 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938c114-c6b3-460b-a002-875a67b7bb34 tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:59.776612 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938c114-c6b3-460b-a002-875a67b7bb34 tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:30:59.776776 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938c114-c6b3-460b-a002-875a67b7bb34 tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:59.778393 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938c114-c6b3-460b-a002-875a67b7bb34 tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:59.793916 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1443/2879] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 15:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:30:59.828983 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf328443-676f-40a5-974e-94cf3d8bf7aa tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:59.839787 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf328443-676f-40a5-974e-94cf3d8bf7aa tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:59.844905 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad1e6e-c493-40de-86cd-500ff70d59de tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9285c91e885a422381e51d253ae4b879', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '565042976fd142dea6924005fbc0ecc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:59.845781 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad1e6e-c493-40de-86cd-500ff70d59de tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:59.846078 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad1e6e-c493-40de-86cd-500ff70d59de tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:59.846356 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad1e6e-c493-40de-86cd-500ff70d59de tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] PATH_INFO: `/v3/projects/8e75c01ddb6f478baf86642be8a1e628` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:59.847588 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad1e6e-c493-40de-86cd-500ff70d59de tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8e75c01ddb6f478baf86642be8a1e628)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:59.849466 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad1e6e-c493-40de-86cd-500ff70d59de tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:59.902238 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-22ad1e6e-c493-40de-86cd-500ff70d59de tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e75c01ddb6f478baf86642be8a1e628', 'request_id': 'req-22ad1e6e-c493-40de-86cd-500ff70d59de'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:59.908244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-22ad1e6e-c493-40de-86cd-500ff70d59de tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e75c01ddb6f478baf86642be8a1e628', 'request_id': 'req-22ad1e6e-c493-40de-86cd-500ff70d59de'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:59.915011 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-22ad1e6e-c493-40de-86cd-500ff70d59de tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e75c01ddb6f478baf86642be8a1e628', 'request_id': 'req-22ad1e6e-c493-40de-86cd-500ff70d59de'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:59.928615 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-22ad1e6e-c493-40de-86cd-500ff70d59de tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] The token cache is being invalidate because project 8e75c01ddb6f478baf86642be8a1e628 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:30:59.928907 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-22ad1e6e-c493-40de-86cd-500ff70d59de tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:59.929560 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-22ad1e6e-c493-40de-86cd-500ff70d59de tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:30:59.931242 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1437/2880] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:30:59 2022] DELETE /identity/v3/projects/8e75c01ddb6f478baf86642be8a1e628 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:00.126415 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938c114-c6b3-460b-a002-875a67b7bb34 tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:00.128205 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-e938c114-c6b3-460b-a002-875a67b7bb34 tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:00.186117 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=100 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:00.206621 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938c114-c6b3-460b-a002-875a67b7bb34 tempest-VolumeQuotaClassesTest-1876051133 tempest-VolumeQuotaClassesTest-1876051133-project-admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:00.285022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29627b7b-fcf2-4633-be66-9e25096b1ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:00.285891 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29627b7b-fcf2-4633-be66-9e25096b1ea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:00.286177 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29627b7b-fcf2-4633-be66-9e25096b1ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:00.286457 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29627b7b-fcf2-4633-be66-9e25096b1ea4 admin admin] PATH_INFO: `/v3/users/7897b4f4a4f842fc8a4677661bbb0315` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:00.288056 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29627b7b-fcf2-4633-be66-9e25096b1ea4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7897b4f4a4f842fc8a4677661bbb0315)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:00.290410 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29627b7b-fcf2-4633-be66-9e25096b1ea4 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:00.348997 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-29627b7b-fcf2-4633-be66-9e25096b1ea4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7897b4f4a4f842fc8a4677661bbb0315'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:00.365326 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-29627b7b-fcf2-4633-be66-9e25096b1ea4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7897b4f4a4f842fc8a4677661bbb0315'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:00.366070 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-29627b7b-fcf2-4633-be66-9e25096b1ea4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7897b4f4a4f842fc8a4677661bbb0315'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:00.366691 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-29627b7b-fcf2-4633-be66-9e25096b1ea4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7897b4f4a4f842fc8a4677661bbb0315'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:00.374169 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-29627b7b-fcf2-4633-be66-9e25096b1ea4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7897b4f4a4f842fc8a4677661bbb0315'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:00.386799 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1444/2881] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:31:00 2022] DELETE /identity/v3/users/7897b4f4a4f842fc8a4677661bbb0315 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:00.391423 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253d2dc-4c0b-4e49-81c8-98fea6b7b2aa None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:00.391897 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253d2dc-4c0b-4e49-81c8-98fea6b7b2aa None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:00.391985 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253d2dc-4c0b-4e49-81c8-98fea6b7b2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:00.409368 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-1253d2dc-4c0b-4e49-81c8-98fea6b7b2aa None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:00.410596 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-1253d2dc-4c0b-4e49-81c8-98fea6b7b2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:00.470872 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1438/2882] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:31:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:00.481725 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29627b7b-fcf2-4633-be66-9e25096b1ea4 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:00.483348 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-29627b7b-fcf2-4633-be66-9e25096b1ea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:00.542644 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29627b7b-fcf2-4633-be66-9e25096b1ea4 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:00.546352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06432131-20c3-481d-9514-b1b799d83aed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:00.547009 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06432131-20c3-481d-9514-b1b799d83aed service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:00.547333 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06432131-20c3-481d-9514-b1b799d83aed service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:00.547615 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06432131-20c3-481d-9514-b1b799d83aed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:00.557159 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06432131-20c3-481d-9514-b1b799d83aed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:00.558594 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06432131-20c3-481d-9514-b1b799d83aed service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:00.568396 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1445/2883] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:31:00.857655 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253d2dc-4c0b-4e49-81c8-98fea6b7b2aa None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:00.859671 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-1253d2dc-4c0b-4e49-81c8-98fea6b7b2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:00.873882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253d2dc-4c0b-4e49-81c8-98fea6b7b2aa None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:00.879738 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90779337-0e94-4598-a5ee-1e01bad54745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:00.880706 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90779337-0e94-4598-a5ee-1e01bad54745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:00.880997 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90779337-0e94-4598-a5ee-1e01bad54745 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:00.881285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90779337-0e94-4598-a5ee-1e01bad54745 admin admin] PATH_INFO: `/v3/projects/0d4e79824ed849e990b0d1540a1de8a3` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:00.882503 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90779337-0e94-4598-a5ee-1e01bad54745 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d4e79824ed849e990b0d1540a1de8a3)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:00.884623 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90779337-0e94-4598-a5ee-1e01bad54745 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:00.899981 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:00.946245 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-90779337-0e94-4598-a5ee-1e01bad54745 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d4e79824ed849e990b0d1540a1de8a3', 'request_id': 'req-90779337-0e94-4598-a5ee-1e01bad54745'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:00.952373 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-90779337-0e94-4598-a5ee-1e01bad54745 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d4e79824ed849e990b0d1540a1de8a3', 'request_id': 'req-90779337-0e94-4598-a5ee-1e01bad54745'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:00.959724 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-90779337-0e94-4598-a5ee-1e01bad54745 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d4e79824ed849e990b0d1540a1de8a3', 'request_id': 'req-90779337-0e94-4598-a5ee-1e01bad54745'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:00.980367 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-90779337-0e94-4598-a5ee-1e01bad54745 admin admin] The token cache is being invalidate because project 0d4e79824ed849e990b0d1540a1de8a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:31:00.980673 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-90779337-0e94-4598-a5ee-1e01bad54745 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:00.981377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-90779337-0e94-4598-a5ee-1e01bad54745 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:00.987497 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06432131-20c3-481d-9514-b1b799d83aed service neutron] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:00.988006 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1439/2884] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:31:00 2022] DELETE /identity/v3/projects/0d4e79824ed849e990b0d1540a1de8a3 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:00.998362 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06432131-20c3-481d-9514-b1b799d83aed service neutron] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:01.012464 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:01.067852 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605cbbb-52b1-45bf-bd50-9731fe03f322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:01.068841 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605cbbb-52b1-45bf-bd50-9731fe03f322 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:01.069193 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605cbbb-52b1-45bf-bd50-9731fe03f322 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:01.069565 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605cbbb-52b1-45bf-bd50-9731fe03f322 admin admin] PATH_INFO: `/v3/users/9285c91e885a422381e51d253ae4b879` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:01.070844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a605cbbb-52b1-45bf-bd50-9731fe03f322 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9285c91e885a422381e51d253ae4b879)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:01.072931 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a605cbbb-52b1-45bf-bd50-9731fe03f322 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:01.132532 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a605cbbb-52b1-45bf-bd50-9731fe03f322 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9285c91e885a422381e51d253ae4b879'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:01.147994 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a605cbbb-52b1-45bf-bd50-9731fe03f322 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9285c91e885a422381e51d253ae4b879'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:01.148733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a605cbbb-52b1-45bf-bd50-9731fe03f322 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9285c91e885a422381e51d253ae4b879'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:01.149397 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a605cbbb-52b1-45bf-bd50-9731fe03f322 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9285c91e885a422381e51d253ae4b879'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:01.158532 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a605cbbb-52b1-45bf-bd50-9731fe03f322 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9285c91e885a422381e51d253ae4b879'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:01.175637 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1446/2885] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:31:00 2022] DELETE /identity/v3/users/9285c91e885a422381e51d253ae4b879 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:01.509024 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90779337-0e94-4598-a5ee-1e01bad54745 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:01.510616 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-90779337-0e94-4598-a5ee-1e01bad54745 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:01.538013 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90779337-0e94-4598-a5ee-1e01bad54745 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:01.616591 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf42c952-4d67-4ad5-9b9c-3968d6e08516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:01.617506 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42c952-4d67-4ad5-9b9c-3968d6e08516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:01.617820 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42c952-4d67-4ad5-9b9c-3968d6e08516 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:01.618102 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42c952-4d67-4ad5-9b9c-3968d6e08516 admin admin] PATH_INFO: `/v3/projects/565042976fd142dea6924005fbc0ecc5` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:01.619536 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf42c952-4d67-4ad5-9b9c-3968d6e08516 admin admin] RBAC: Authorizing `identity:delete_project(project_id=565042976fd142dea6924005fbc0ecc5)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:01.621747 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf42c952-4d67-4ad5-9b9c-3968d6e08516 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:01.683959 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-cf42c952-4d67-4ad5-9b9c-3968d6e08516 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '565042976fd142dea6924005fbc0ecc5', 'request_id': 'req-cf42c952-4d67-4ad5-9b9c-3968d6e08516'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:01.696352 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-cf42c952-4d67-4ad5-9b9c-3968d6e08516 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '565042976fd142dea6924005fbc0ecc5', 'request_id': 'req-cf42c952-4d67-4ad5-9b9c-3968d6e08516'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:01.702773 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-cf42c952-4d67-4ad5-9b9c-3968d6e08516 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '565042976fd142dea6924005fbc0ecc5', 'request_id': 'req-cf42c952-4d67-4ad5-9b9c-3968d6e08516'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:01.716247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-cf42c952-4d67-4ad5-9b9c-3968d6e08516 admin admin] The token cache is being invalidate because project 565042976fd142dea6924005fbc0ecc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:31:01.716540 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-cf42c952-4d67-4ad5-9b9c-3968d6e08516 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:01.717179 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-cf42c952-4d67-4ad5-9b9c-3968d6e08516 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:01.718749 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1440/2886] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:31:01 2022] DELETE /identity/v3/projects/565042976fd142dea6924005fbc0ecc5 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:01.723893 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998920d4-6c01-4515-bd0c-f9c7963a233a None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:01.724253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998920d4-6c01-4515-bd0c-f9c7963a233a None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:01.724606 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998920d4-6c01-4515-bd0c-f9c7963a233a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:01.746443 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-998920d4-6c01-4515-bd0c-f9c7963a233a None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:01.748008 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-998920d4-6c01-4515-bd0c-f9c7963a233a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:01.852789 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1447/2887] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:31:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:01.860880 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad968c0-1264-4256-8693-3d9fa98e5e29 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:01.861166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad968c0-1264-4256-8693-3d9fa98e5e29 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:01.861444 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad968c0-1264-4256-8693-3d9fa98e5e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:01.880947 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-5ad968c0-1264-4256-8693-3d9fa98e5e29 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:01.882174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-5ad968c0-1264-4256-8693-3d9fa98e5e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:01.891093 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1441/2888] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:31:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:01.894943 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998920d4-6c01-4515-bd0c-f9c7963a233a None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:01.908044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998920d4-6c01-4515-bd0c-f9c7963a233a None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:01.915072 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5de53-8778-4512-8ab6-d6f301742df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:01.915690 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5de53-8778-4512-8ab6-d6f301742df0 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:01.915996 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5de53-8778-4512-8ab6-d6f301742df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:01.916272 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5de53-8778-4512-8ab6-d6f301742df0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:01.918648 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c5de53-8778-4512-8ab6-d6f301742df0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:31:01.918648 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c5de53-8778-4512-8ab6-d6f301742df0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:01.918978 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c5de53-8778-4512-8ab6-d6f301742df0 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:01.938306 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1448/2889] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 15:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:31:01.939040 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad968c0-1264-4256-8693-3d9fa98e5e29 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:01.951381 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad968c0-1264-4256-8693-3d9fa98e5e29 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:01.959835 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed1c44-f3a3-47ed-887b-12e08cc74c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:01.960532 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed1c44-f3a3-47ed-887b-12e08cc74c5a admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:01.960812 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed1c44-f3a3-47ed-887b-12e08cc74c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:01.961090 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed1c44-f3a3-47ed-887b-12e08cc74c5a admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:01.962085 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deed1c44-f3a3-47ed-887b-12e08cc74c5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:01.964098 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deed1c44-f3a3-47ed-887b-12e08cc74c5a admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:02.011075 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1442/2890] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:31:01 2022] POST /identity/v3/projects => generated 385 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:31:02.012997 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5de53-8778-4512-8ab6-d6f301742df0 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:02.023669 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5de53-8778-4512-8ab6-d6f301742df0 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:02.067278 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:02.123025 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42401dcb-d8b5-4168-996e-ec9fb63d8602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:02.123787 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42401dcb-d8b5-4168-996e-ec9fb63d8602 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:02.124345 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42401dcb-d8b5-4168-996e-ec9fb63d8602 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:02.124624 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42401dcb-d8b5-4168-996e-ec9fb63d8602 admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:02.125695 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42401dcb-d8b5-4168-996e-ec9fb63d8602 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:02.128094 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42401dcb-d8b5-4168-996e-ec9fb63d8602 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:02.166948 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1449/2891] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:31:02 2022] POST /identity/v3/users => generated 293 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:31:02.168397 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed1c44-f3a3-47ed-887b-12e08cc74c5a admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:02.177200 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:02.193189 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed1c44-f3a3-47ed-887b-12e08cc74c5a admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:02.199417 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4517a889-7f33-4c65-858e-94d15ac5f801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:02.199936 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517a889-7f33-4c65-858e-94d15ac5f801 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:02.200348 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517a889-7f33-4c65-858e-94d15ac5f801 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:02.200795 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517a889-7f33-4c65-858e-94d15ac5f801 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:02.201601 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4517a889-7f33-4c65-858e-94d15ac5f801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:02.204221 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4517a889-7f33-4c65-858e-94d15ac5f801 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:02.230501 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1443/2892] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:31:02.234618 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42401dcb-d8b5-4168-996e-ec9fb63d8602 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:02.246655 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42401dcb-d8b5-4168-996e-ec9fb63d8602 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:02.252472 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a9874-da87-434c-8e00-c4d613ab62b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:02.253648 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a9874-da87-434c-8e00-c4d613ab62b6 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:02.253648 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a9874-da87-434c-8e00-c4d613ab62b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:02.254043 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a9874-da87-434c-8e00-c4d613ab62b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:02.254854 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949a9874-da87-434c-8e00-c4d613ab62b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:02.256564 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949a9874-da87-434c-8e00-c4d613ab62b6 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:02.274300 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1450/2893] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:31:02.275757 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4517a889-7f33-4c65-858e-94d15ac5f801 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:02.287077 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4517a889-7f33-4c65-858e-94d15ac5f801 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:02.292446 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff8130-6bc4-49e7-bd92-4aeda62190b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:02.293363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff8130-6bc4-49e7-bd92-4aeda62190b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:02.293660 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff8130-6bc4-49e7-bd92-4aeda62190b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:02.293987 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff8130-6bc4-49e7-bd92-4aeda62190b5 admin admin] PATH_INFO: `/v3/projects/cf70207142e74dc0b01eb6fe773c470a/users/6b46613c2c9b4e8b9a3c4cd3b4b99d2d/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:02.310196 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff8130-6bc4-49e7-bd92-4aeda62190b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf70207142e74dc0b01eb6fe773c470a, user_id=6b46613c2c9b4e8b9a3c4cd3b4b99d2d, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:02.312937 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff8130-6bc4-49e7-bd92-4aeda62190b5 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:02.329013 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1444/2894] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:31:02 2022] PUT /identity/v3/projects/cf70207142e74dc0b01eb6fe773c470a/users/6b46613c2c9b4e8b9a3c4cd3b4b99d2d/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:02.385176 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a9874-da87-434c-8e00-c4d613ab62b6 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:02.387120 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-949a9874-da87-434c-8e00-c4d613ab62b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:02.399779 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff2854-44c3-4061-a022-ab1b91d42d68 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:02.400067 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff2854-44c3-4061-a022-ab1b91d42d68 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:02.400346 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff2854-44c3-4061-a022-ab1b91d42d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:02.430485 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-76ff2854-44c3-4061-a022-ab1b91d42d68 None None] MFA Rules not processed for user `6b46613c2c9b4e8b9a3c4cd3b4b99d2d`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:02.432096 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-76ff2854-44c3-4061-a022-ab1b91d42d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:02.484892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a9874-da87-434c-8e00-c4d613ab62b6 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:02.531448 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1445/2895] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:31:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:31:02.536045 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ff2854-44c3-4061-a022-ab1b91d42d68 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:02.573203 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977a90b-4dbd-429a-9152-a7ef2e8a088a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb857f32bbc04d82bfa681bd5659765f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:02.574294 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977a90b-4dbd-429a-9152-a7ef2e8a088a service placement] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:02.574667 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977a90b-4dbd-429a-9152-a7ef2e8a088a service placement] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:02.574946 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977a90b-4dbd-429a-9152-a7ef2e8a088a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:02.576965 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-3977a90b-4dbd-429a-9152-a7ef2e8a088a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:02.590352 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3977a90b-4dbd-429a-9152-a7ef2e8a088a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:02.592171 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3977a90b-4dbd-429a-9152-a7ef2e8a088a service placement] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:02.603560 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1451/2896] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:31:02.609247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ff2854-44c3-4061-a022-ab1b91d42d68 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:02.614247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e025932b-f600-4399-ade9-c003dbcc5e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:02.614919 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025932b-f600-4399-ade9-c003dbcc5e73 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:02.615238 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025932b-f600-4399-ade9-c003dbcc5e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:02.615970 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025932b-f600-4399-ade9-c003dbcc5e73 admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:02.616840 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e025932b-f600-4399-ade9-c003dbcc5e73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:02.618907 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e025932b-f600-4399-ade9-c003dbcc5e73 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:02.647764 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1446/2897] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:31:02 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:31:02.651846 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977a90b-4dbd-429a-9152-a7ef2e8a088a service placement] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:02.663367 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977a90b-4dbd-429a-9152-a7ef2e8a088a service placement] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:02.696454 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:02.725961 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4918c6-1b65-40e8-9e54-dee35620045d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:02.726706 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4918c6-1b65-40e8-9e54-dee35620045d admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:02.727380 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4918c6-1b65-40e8-9e54-dee35620045d admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:02.727697 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4918c6-1b65-40e8-9e54-dee35620045d admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:02.728468 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4918c6-1b65-40e8-9e54-dee35620045d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:02.730444 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4918c6-1b65-40e8-9e54-dee35620045d admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:02.754476 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1452/2898] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:31:02 2022] POST /identity/v3/users => generated 298 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:31:02.759288 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e025932b-f600-4399-ade9-c003dbcc5e73 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:02.775101 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e025932b-f600-4399-ade9-c003dbcc5e73 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:02.780190 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de8739-08cd-44a9-b9ab-69fc4b4bac62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:02.780873 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de8739-08cd-44a9-b9ab-69fc4b4bac62 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:02.781163 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de8739-08cd-44a9-b9ab-69fc4b4bac62 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:02.781443 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de8739-08cd-44a9-b9ab-69fc4b4bac62 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:02.782187 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70de8739-08cd-44a9-b9ab-69fc4b4bac62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:02.783847 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70de8739-08cd-44a9-b9ab-69fc4b4bac62 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:02.795553 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1447/2899] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:31:02.800234 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4918c6-1b65-40e8-9e54-dee35620045d admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:02.811965 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4918c6-1b65-40e8-9e54-dee35620045d admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:02.817429 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5087e51-e3d6-41c8-ae65-17a8950592cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:02.818429 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5087e51-e3d6-41c8-ae65-17a8950592cf admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:02.818712 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5087e51-e3d6-41c8-ae65-17a8950592cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:02.818997 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5087e51-e3d6-41c8-ae65-17a8950592cf admin admin] PATH_INFO: `/v3/projects/a6e6337ec9404b2c8df92e08d14a7750/users/8d456c9e5262419286d995b2326f0612/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:02.835475 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5087e51-e3d6-41c8-ae65-17a8950592cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6e6337ec9404b2c8df92e08d14a7750, user_id=8d456c9e5262419286d995b2326f0612, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:02.838398 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5087e51-e3d6-41c8-ae65-17a8950592cf admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:02.854820 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1453/2900] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:31:02 2022] PUT /identity/v3/projects/a6e6337ec9404b2c8df92e08d14a7750/users/8d456c9e5262419286d995b2326f0612/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:02.858889 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de8739-08cd-44a9-b9ab-69fc4b4bac62 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:02.871171 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de8739-08cd-44a9-b9ab-69fc4b4bac62 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:02.903844 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:02.942116 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a13161-e0c6-4407-9968-fdd899bb7873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:02.943033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a13161-e0c6-4407-9968-fdd899bb7873 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:02.943033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a13161-e0c6-4407-9968-fdd899bb7873 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:02.943467 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a13161-e0c6-4407-9968-fdd899bb7873 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:02.944736 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a13161-e0c6-4407-9968-fdd899bb7873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:02.946745 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a13161-e0c6-4407-9968-fdd899bb7873 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:02.957370 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1448/2901] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:31:02.961989 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5087e51-e3d6-41c8-ae65-17a8950592cf admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:02.975530 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5087e51-e3d6-41c8-ae65-17a8950592cf admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:02.983642 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd81926-e5ba-4e94-b1e4-9cbde1da832c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:02.985082 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd81926-e5ba-4e94-b1e4-9cbde1da832c admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:02.985526 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd81926-e5ba-4e94-b1e4-9cbde1da832c admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:02.985965 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd81926-e5ba-4e94-b1e4-9cbde1da832c admin admin] PATH_INFO: `/v3/projects/a6e6337ec9404b2c8df92e08d14a7750/users/8d456c9e5262419286d995b2326f0612/roles/49bd5abf255048a0abaa9538e309f516` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:02.989142 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd81926-e5ba-4e94-b1e4-9cbde1da832c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6e6337ec9404b2c8df92e08d14a7750, user_id=8d456c9e5262419286d995b2326f0612, role_id=49bd5abf255048a0abaa9538e309f516)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:02.994053 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd81926-e5ba-4e94-b1e4-9cbde1da832c admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:03.010928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1454/2902] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:31:02 2022] PUT /identity/v3/projects/a6e6337ec9404b2c8df92e08d14a7750/users/8d456c9e5262419286d995b2326f0612/roles/49bd5abf255048a0abaa9538e309f516 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:31:03.021541 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a25cc-a0c6-469b-8f20-1afb30e14c63 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:03.021833 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a25cc-a0c6-469b-8f20-1afb30e14c63 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:03.022116 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a25cc-a0c6-469b-8f20-1afb30e14c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:03.051150 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-a30a25cc-a0c6-469b-8f20-1afb30e14c63 None None] MFA Rules not processed for user `8d456c9e5262419286d995b2326f0612`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:03.052394 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-a30a25cc-a0c6-469b-8f20-1afb30e14c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:03.145811 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1449/2903] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:31:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:03.156749 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd81926-e5ba-4e94-b1e4-9cbde1da832c admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:03.158563 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-cfd81926-e5ba-4e94-b1e4-9cbde1da832c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:03.221160 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd81926-e5ba-4e94-b1e4-9cbde1da832c admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:03.225664 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d005608f-e019-428c-aa09-87f09eea0527 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:03.226350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005608f-e019-428c-aa09-87f09eea0527 service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:03.226630 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005608f-e019-428c-aa09-87f09eea0527 service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:03.227297 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005608f-e019-428c-aa09-87f09eea0527 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:03.286473 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d005608f-e019-428c-aa09-87f09eea0527 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:03.287982 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d005608f-e019-428c-aa09-87f09eea0527 service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:03.297196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1455/2904] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:31:06.078927 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a25cc-a0c6-469b-8f20-1afb30e14c63 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:06.081222 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-a30a25cc-a0c6-469b-8f20-1afb30e14c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:06.158691 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a25cc-a0c6-469b-8f20-1afb30e14c63 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:06.163489 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4864b6b9-20d0-40e4-a75d-6f1d35efe98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:06.165290 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864b6b9-20d0-40e4-a75d-6f1d35efe98b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:06.165290 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864b6b9-20d0-40e4-a75d-6f1d35efe98b admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:06.165290 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864b6b9-20d0-40e4-a75d-6f1d35efe98b admin admin] PATH_INFO: `/v3/users/9b70cbf2ad4f405b84cbea240b83cd05` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:06.167807 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4864b6b9-20d0-40e4-a75d-6f1d35efe98b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b70cbf2ad4f405b84cbea240b83cd05)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:06.169553 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4864b6b9-20d0-40e4-a75d-6f1d35efe98b admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:06.230642 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-4864b6b9-20d0-40e4-a75d-6f1d35efe98b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b70cbf2ad4f405b84cbea240b83cd05'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:06.235934 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:06.244886 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-4864b6b9-20d0-40e4-a75d-6f1d35efe98b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b70cbf2ad4f405b84cbea240b83cd05'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:06.245728 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-4864b6b9-20d0-40e4-a75d-6f1d35efe98b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b70cbf2ad4f405b84cbea240b83cd05'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:06.246335 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-4864b6b9-20d0-40e4-a75d-6f1d35efe98b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b70cbf2ad4f405b84cbea240b83cd05'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:06.253703 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-4864b6b9-20d0-40e4-a75d-6f1d35efe98b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b70cbf2ad4f405b84cbea240b83cd05'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:06.275098 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1450/2905] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:31:06 2022] DELETE /identity/v3/users/9b70cbf2ad4f405b84cbea240b83cd05 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:06.465734 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d005608f-e019-428c-aa09-87f09eea0527 service cinder] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:06.479439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d005608f-e019-428c-aa09-87f09eea0527 service cinder] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:06.546873 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:06.568000 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3032b6e-a2b7-4480-9df9-49c396dfc13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:06.568883 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3032b6e-a2b7-4480-9df9-49c396dfc13f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:06.569165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3032b6e-a2b7-4480-9df9-49c396dfc13f admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:06.569445 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3032b6e-a2b7-4480-9df9-49c396dfc13f admin admin] PATH_INFO: `/v3/projects/535d117f782341d5aa579a8fb34c35c5` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:06.570643 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3032b6e-a2b7-4480-9df9-49c396dfc13f admin admin] RBAC: Authorizing `identity:delete_project(project_id=535d117f782341d5aa579a8fb34c35c5)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:06.572715 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3032b6e-a2b7-4480-9df9-49c396dfc13f admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:06.634006 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-d3032b6e-a2b7-4480-9df9-49c396dfc13f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '535d117f782341d5aa579a8fb34c35c5', 'request_id': 'req-d3032b6e-a2b7-4480-9df9-49c396dfc13f'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:06.639835 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-d3032b6e-a2b7-4480-9df9-49c396dfc13f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '535d117f782341d5aa579a8fb34c35c5', 'request_id': 'req-d3032b6e-a2b7-4480-9df9-49c396dfc13f'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:06.648208 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-d3032b6e-a2b7-4480-9df9-49c396dfc13f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '535d117f782341d5aa579a8fb34c35c5', 'request_id': 'req-d3032b6e-a2b7-4480-9df9-49c396dfc13f'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:06.659948 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-d3032b6e-a2b7-4480-9df9-49c396dfc13f admin admin] The token cache is being invalidate because project 535d117f782341d5aa579a8fb34c35c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:31:06.660227 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-d3032b6e-a2b7-4480-9df9-49c396dfc13f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:06.660848 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-d3032b6e-a2b7-4480-9df9-49c396dfc13f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:06.662431 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1456/2906] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:31:06 2022] DELETE /identity/v3/projects/535d117f782341d5aa579a8fb34c35c5 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:06.666556 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4864b6b9-20d0-40e4-a75d-6f1d35efe98b admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:06.668130 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-4864b6b9-20d0-40e4-a75d-6f1d35efe98b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:06.682464 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4864b6b9-20d0-40e4-a75d-6f1d35efe98b admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:06.749976 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c84de-7ed9-49ec-8a60-016c4fe44122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:06.750864 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c84de-7ed9-49ec-8a60-016c4fe44122 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:06.751184 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c84de-7ed9-49ec-8a60-016c4fe44122 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:06.751466 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c84de-7ed9-49ec-8a60-016c4fe44122 admin admin] PATH_INFO: `/v3/users/2ef9fd651da04b51bef1ad1af365fba5` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:06.752837 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c84de-7ed9-49ec-8a60-016c4fe44122 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ef9fd651da04b51bef1ad1af365fba5)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:06.754792 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c84de-7ed9-49ec-8a60-016c4fe44122 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:06.815431 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-654c84de-7ed9-49ec-8a60-016c4fe44122 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ef9fd651da04b51bef1ad1af365fba5'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:06.827703 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-654c84de-7ed9-49ec-8a60-016c4fe44122 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ef9fd651da04b51bef1ad1af365fba5'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:06.828402 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-654c84de-7ed9-49ec-8a60-016c4fe44122 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ef9fd651da04b51bef1ad1af365fba5'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:06.828986 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-654c84de-7ed9-49ec-8a60-016c4fe44122 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ef9fd651da04b51bef1ad1af365fba5'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:06.835775 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-654c84de-7ed9-49ec-8a60-016c4fe44122 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ef9fd651da04b51bef1ad1af365fba5'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:06.851127 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1451/2907] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:31:06 2022] DELETE /identity/v3/users/2ef9fd651da04b51bef1ad1af365fba5 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:07.047120 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3032b6e-a2b7-4480-9df9-49c396dfc13f admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:07.049100 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-d3032b6e-a2b7-4480-9df9-49c396dfc13f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:07.062344 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3032b6e-a2b7-4480-9df9-49c396dfc13f admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:07.127895 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82086697-93a6-4f43-b1df-c2f053ee6f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:07.128822 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82086697-93a6-4f43-b1df-c2f053ee6f02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:07.129101 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82086697-93a6-4f43-b1df-c2f053ee6f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:07.129401 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82086697-93a6-4f43-b1df-c2f053ee6f02 admin admin] PATH_INFO: `/v3/projects/84297365b01d4761a82314a1aff95c38` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:07.130761 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82086697-93a6-4f43-b1df-c2f053ee6f02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84297365b01d4761a82314a1aff95c38)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:07.132733 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82086697-93a6-4f43-b1df-c2f053ee6f02 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:07.183292 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-82086697-93a6-4f43-b1df-c2f053ee6f02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84297365b01d4761a82314a1aff95c38', 'request_id': 'req-82086697-93a6-4f43-b1df-c2f053ee6f02'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:07.189439 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-82086697-93a6-4f43-b1df-c2f053ee6f02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84297365b01d4761a82314a1aff95c38', 'request_id': 'req-82086697-93a6-4f43-b1df-c2f053ee6f02'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:07.197558 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-82086697-93a6-4f43-b1df-c2f053ee6f02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84297365b01d4761a82314a1aff95c38', 'request_id': 'req-82086697-93a6-4f43-b1df-c2f053ee6f02'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:07.206120 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:07.210945 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-82086697-93a6-4f43-b1df-c2f053ee6f02 admin admin] The token cache is being invalidate because project 84297365b01d4761a82314a1aff95c38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:31:07.211530 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-82086697-93a6-4f43-b1df-c2f053ee6f02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:07.212622 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-82086697-93a6-4f43-b1df-c2f053ee6f02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:07.214464 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1457/2908] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:31:07 2022] DELETE /identity/v3/projects/84297365b01d4761a82314a1aff95c38 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:07.664746 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c84de-7ed9-49ec-8a60-016c4fe44122 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:07.666424 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-654c84de-7ed9-49ec-8a60-016c4fe44122 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:07.725165 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c84de-7ed9-49ec-8a60-016c4fe44122 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:07.729605 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:07.789035 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad7eea-0b08-4e03-8625-9702e6186d68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:07.789709 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad7eea-0b08-4e03-8625-9702e6186d68 service cinder] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:07.789996 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad7eea-0b08-4e03-8625-9702e6186d68 service cinder] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:07.790258 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad7eea-0b08-4e03-8625-9702e6186d68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:07.792448 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-fbad7eea-0b08-4e03-8625-9702e6186d68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:07.855978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbad7eea-0b08-4e03-8625-9702e6186d68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:07.857418 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbad7eea-0b08-4e03-8625-9702e6186d68 service cinder] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:07.867164 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1452/2909] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:07 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:31:12.799501 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82086697-93a6-4f43-b1df-c2f053ee6f02 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:12.801064 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-82086697-93a6-4f43-b1df-c2f053ee6f02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:12.873378 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82086697-93a6-4f43-b1df-c2f053ee6f02 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:12.880185 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d441d-cf62-45fa-8d3a-50b166b823ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:12.881414 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d441d-cf62-45fa-8d3a-50b166b823ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:12.881781 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d441d-cf62-45fa-8d3a-50b166b823ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:12.882217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d441d-cf62-45fa-8d3a-50b166b823ae admin admin] PATH_INFO: `/v3/users/6b46613c2c9b4e8b9a3c4cd3b4b99d2d` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:12.883929 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8d441d-cf62-45fa-8d3a-50b166b823ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b46613c2c9b4e8b9a3c4cd3b4b99d2d)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:12.885917 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8d441d-cf62-45fa-8d3a-50b166b823ae admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:12.939963 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-8d8d441d-cf62-45fa-8d3a-50b166b823ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b46613c2c9b4e8b9a3c4cd3b4b99d2d'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:12.955226 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-8d8d441d-cf62-45fa-8d3a-50b166b823ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b46613c2c9b4e8b9a3c4cd3b4b99d2d'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:12.956047 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-8d8d441d-cf62-45fa-8d3a-50b166b823ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b46613c2c9b4e8b9a3c4cd3b4b99d2d'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:12.956791 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-8d8d441d-cf62-45fa-8d3a-50b166b823ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b46613c2c9b4e8b9a3c4cd3b4b99d2d'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:12.965253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-8d8d441d-cf62-45fa-8d3a-50b166b823ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b46613c2c9b4e8b9a3c4cd3b4b99d2d'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:12.983232 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1458/2910] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:31:12 2022] DELETE /identity/v3/users/6b46613c2c9b4e8b9a3c4cd3b4b99d2d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:12.987934 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe882b7-9fd7-4abb-aceb-1ba5be128c82 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:12.988454 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe882b7-9fd7-4abb-aceb-1ba5be128c82 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:12.988555 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe882b7-9fd7-4abb-aceb-1ba5be128c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:13.010262 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-8fe882b7-9fd7-4abb-aceb-1ba5be128c82 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:13.011581 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-8fe882b7-9fd7-4abb-aceb-1ba5be128c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:13.075903 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1453/2911] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:31:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:31:13.085775 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d441d-cf62-45fa-8d3a-50b166b823ae admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:13.087638 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-8d8d441d-cf62-45fa-8d3a-50b166b823ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:13.150129 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d441d-cf62-45fa-8d3a-50b166b823ae admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:13.154599 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f9181-3130-4bf2-beca-027d121ac845 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:13.155286 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f9181-3130-4bf2-beca-027d121ac845 service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:13.155573 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f9181-3130-4bf2-beca-027d121ac845 service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:13.155848 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f9181-3130-4bf2-beca-027d121ac845 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:13.168090 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f9181-3130-4bf2-beca-027d121ac845 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:13.169583 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f9181-3130-4bf2-beca-027d121ac845 service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:13.179339 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1459/2912] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:31:13.480227 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe882b7-9fd7-4abb-aceb-1ba5be128c82 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:13.482029 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-8fe882b7-9fd7-4abb-aceb-1ba5be128c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:13.497275 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe882b7-9fd7-4abb-aceb-1ba5be128c82 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:13.502368 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e684013c-c02c-48f2-8438-6551fc285522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:13.503403 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684013c-c02c-48f2-8438-6551fc285522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:13.503693 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684013c-c02c-48f2-8438-6551fc285522 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:13.503973 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684013c-c02c-48f2-8438-6551fc285522 admin admin] PATH_INFO: `/v3/projects/cf70207142e74dc0b01eb6fe773c470a` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:13.505839 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e684013c-c02c-48f2-8438-6551fc285522 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf70207142e74dc0b01eb6fe773c470a)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:13.508379 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e684013c-c02c-48f2-8438-6551fc285522 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:13.560349 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-e684013c-c02c-48f2-8438-6551fc285522 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf70207142e74dc0b01eb6fe773c470a', 'request_id': 'req-e684013c-c02c-48f2-8438-6551fc285522'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:13.562378 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:13.567958 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-e684013c-c02c-48f2-8438-6551fc285522 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf70207142e74dc0b01eb6fe773c470a', 'request_id': 'req-e684013c-c02c-48f2-8438-6551fc285522'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:13.575419 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-e684013c-c02c-48f2-8438-6551fc285522 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf70207142e74dc0b01eb6fe773c470a', 'request_id': 'req-e684013c-c02c-48f2-8438-6551fc285522'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:13.589887 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-e684013c-c02c-48f2-8438-6551fc285522 admin admin] The token cache is being invalidate because project cf70207142e74dc0b01eb6fe773c470a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:31:13.590174 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-e684013c-c02c-48f2-8438-6551fc285522 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:13.590809 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-e684013c-c02c-48f2-8438-6551fc285522 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:13.592375 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1454/2913] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:31:13 2022] DELETE /identity/v3/projects/cf70207142e74dc0b01eb6fe773c470a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:13.596527 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f9181-3130-4bf2-beca-027d121ac845 service neutron] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:13.607293 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f9181-3130-4bf2-beca-027d121ac845 service neutron] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:13.629384 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:13.672702 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a806ae7-78e5-453b-bb18-49ba24d55b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:13.673593 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a806ae7-78e5-453b-bb18-49ba24d55b26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:13.673871 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a806ae7-78e5-453b-bb18-49ba24d55b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:13.674146 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a806ae7-78e5-453b-bb18-49ba24d55b26 admin admin] PATH_INFO: `/v3/users/8d456c9e5262419286d995b2326f0612` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:13.675409 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a806ae7-78e5-453b-bb18-49ba24d55b26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d456c9e5262419286d995b2326f0612)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:13.677350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a806ae7-78e5-453b-bb18-49ba24d55b26 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:13.727785 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-8a806ae7-78e5-453b-bb18-49ba24d55b26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d456c9e5262419286d995b2326f0612'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:13.740109 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-8a806ae7-78e5-453b-bb18-49ba24d55b26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d456c9e5262419286d995b2326f0612'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:13.740851 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-8a806ae7-78e5-453b-bb18-49ba24d55b26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d456c9e5262419286d995b2326f0612'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:13.741471 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-8a806ae7-78e5-453b-bb18-49ba24d55b26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d456c9e5262419286d995b2326f0612'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:13.748648 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-8a806ae7-78e5-453b-bb18-49ba24d55b26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d456c9e5262419286d995b2326f0612'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:13.767379 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1460/2914] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:31:13 2022] DELETE /identity/v3/users/8d456c9e5262419286d995b2326f0612 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:14.069834 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e684013c-c02c-48f2-8438-6551fc285522 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:14.071460 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e684013c-c02c-48f2-8438-6551fc285522 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:14.087289 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e684013c-c02c-48f2-8438-6551fc285522 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:14.159602 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dde6eb-a751-47cd-8225-d40fc21f696e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:14.160468 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dde6eb-a751-47cd-8225-d40fc21f696e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:14.160716 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dde6eb-a751-47cd-8225-d40fc21f696e admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:14.161017 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dde6eb-a751-47cd-8225-d40fc21f696e admin admin] PATH_INFO: `/v3/projects/a6e6337ec9404b2c8df92e08d14a7750` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:14.162232 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dde6eb-a751-47cd-8225-d40fc21f696e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6e6337ec9404b2c8df92e08d14a7750)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:14.164191 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dde6eb-a751-47cd-8225-d40fc21f696e admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:14.214509 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-77dde6eb-a751-47cd-8225-d40fc21f696e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6e6337ec9404b2c8df92e08d14a7750', 'request_id': 'req-77dde6eb-a751-47cd-8225-d40fc21f696e'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:14.220312 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-77dde6eb-a751-47cd-8225-d40fc21f696e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6e6337ec9404b2c8df92e08d14a7750', 'request_id': 'req-77dde6eb-a751-47cd-8225-d40fc21f696e'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:14.227425 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-77dde6eb-a751-47cd-8225-d40fc21f696e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6e6337ec9404b2c8df92e08d14a7750', 'request_id': 'req-77dde6eb-a751-47cd-8225-d40fc21f696e'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:14.244545 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-77dde6eb-a751-47cd-8225-d40fc21f696e admin admin] The token cache is being invalidate because project a6e6337ec9404b2c8df92e08d14a7750 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:31:14.244808 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-77dde6eb-a751-47cd-8225-d40fc21f696e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:14.245500 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-77dde6eb-a751-47cd-8225-d40fc21f696e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:14.247125 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1455/2915] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:31:14 2022] DELETE /identity/v3/projects/a6e6337ec9404b2c8df92e08d14a7750 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:14.253403 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2172a1-b51e-4370-81d2-7973e668df25 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:14.253933 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2172a1-b51e-4370-81d2-7973e668df25 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:14.254023 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2172a1-b51e-4370-81d2-7973e668df25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:14.273442 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-9c2172a1-b51e-4370-81d2-7973e668df25 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:14.274762 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-9c2172a1-b51e-4370-81d2-7973e668df25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:14.339217 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1461/2916] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:31:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:14.352835 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9100f0-0c17-4103-ae84-15c256439436 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:14.353027 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9100f0-0c17-4103-ae84-15c256439436 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:14.353358 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9100f0-0c17-4103-ae84-15c256439436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:14.372535 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-ee9100f0-0c17-4103-ae84-15c256439436 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:14.373755 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-ee9100f0-0c17-4103-ae84-15c256439436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:14.382518 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1456/2917] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:31:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:14.387467 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2172a1-b51e-4370-81d2-7973e668df25 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:14.399533 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2172a1-b51e-4370-81d2-7973e668df25 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:14.405210 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01efd565-993b-4b77-be43-f54d88f2add2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:14.405915 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efd565-993b-4b77-be43-f54d88f2add2 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:14.406200 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efd565-993b-4b77-be43-f54d88f2add2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:14.406478 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efd565-993b-4b77-be43-f54d88f2add2 admin admin] PATH_INFO: `/v3/domains` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:14.407249 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01efd565-993b-4b77-be43-f54d88f2add2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:31:14.407555 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01efd565-993b-4b77-be43-f54d88f2add2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:14.409179 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01efd565-993b-4b77-be43-f54d88f2add2 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:14.421459 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1462/2918] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 15:31:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:31:14.426042 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9100f0-0c17-4103-ae84-15c256439436 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:14.436043 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9100f0-0c17-4103-ae84-15c256439436 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:14.441676 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b6dba-e338-4b92-b4f3-8d6b8f10a6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:14.442377 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b6dba-e338-4b92-b4f3-8d6b8f10a6ac admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:14.442747 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b6dba-e338-4b92-b4f3-8d6b8f10a6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:14.442971 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b6dba-e338-4b92-b4f3-8d6b8f10a6ac admin admin] PATH_INFO: `/v3/projects` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:14.443715 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137b6dba-e338-4b92-b4f3-8d6b8f10a6ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:14.445801 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137b6dba-e338-4b92-b4f3-8d6b8f10a6ac admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:14.483791 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1457/2919] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:31:14 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:31:14.485526 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01efd565-993b-4b77-be43-f54d88f2add2 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:14.499234 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01efd565-993b-4b77-be43-f54d88f2add2 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:14.535800 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:14.573417 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09a31b-0b8e-429a-86dc-62735fbb1f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:14.574096 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09a31b-0b8e-429a-86dc-62735fbb1f4e admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:14.574383 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09a31b-0b8e-429a-86dc-62735fbb1f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:14.574662 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09a31b-0b8e-429a-86dc-62735fbb1f4e admin admin] PATH_INFO: `/v3/users` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:14.575424 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b09a31b-0b8e-429a-86dc-62735fbb1f4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:14.577350 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b09a31b-0b8e-429a-86dc-62735fbb1f4e admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:14.603894 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1463/2920] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:31:14 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:31:14.608046 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b6dba-e338-4b92-b4f3-8d6b8f10a6ac admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:14.620913 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b6dba-e338-4b92-b4f3-8d6b8f10a6ac admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:14.628003 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f1cd6-b305-46f3-9547-ee4fa50821a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:14.628713 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f1cd6-b305-46f3-9547-ee4fa50821a1 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:14.628973 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f1cd6-b305-46f3-9547-ee4fa50821a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:14.629363 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f1cd6-b305-46f3-9547-ee4fa50821a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:14.629997 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f1cd6-b305-46f3-9547-ee4fa50821a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:14.631688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f1cd6-b305-46f3-9547-ee4fa50821a1 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:14.643911 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1458/2921] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:31:14.648227 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09a31b-0b8e-429a-86dc-62735fbb1f4e admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:14.659355 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09a31b-0b8e-429a-86dc-62735fbb1f4e admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:14.665406 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724dcb91-e1ab-4fdc-8ba8-6e9dfb72eeb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:14.666165 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724dcb91-e1ab-4fdc-8ba8-6e9dfb72eeb3 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:14.666493 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724dcb91-e1ab-4fdc-8ba8-6e9dfb72eeb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:14.666825 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724dcb91-e1ab-4fdc-8ba8-6e9dfb72eeb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:14.667688 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724dcb91-e1ab-4fdc-8ba8-6e9dfb72eeb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:14.669353 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724dcb91-e1ab-4fdc-8ba8-6e9dfb72eeb3 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:14.681228 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1464/2922] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:31:14.685901 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f1cd6-b305-46f3-9547-ee4fa50821a1 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:14.697310 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f1cd6-b305-46f3-9547-ee4fa50821a1 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:14.705501 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba9ab82-448e-4578-871e-beb746f08a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:14.706423 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9ab82-448e-4578-871e-beb746f08a33 admin admin] REQUEST_METHOD: `PUT` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:14.706704 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9ab82-448e-4578-871e-beb746f08a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:14.706984 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9ab82-448e-4578-871e-beb746f08a33 admin admin] PATH_INFO: `/v3/projects/051a8fff39f0432889e36fc60ab02702/users/1ea733c4fd664a46bf4d959e8dac1f4e/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:14.723561 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba9ab82-448e-4578-871e-beb746f08a33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=051a8fff39f0432889e36fc60ab02702, user_id=1ea733c4fd664a46bf4d959e8dac1f4e, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:14.726244 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba9ab82-448e-4578-871e-beb746f08a33 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:14.740845 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1459/2923] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:31:14 2022] PUT /identity/v3/projects/051a8fff39f0432889e36fc60ab02702/users/1ea733c4fd664a46bf4d959e8dac1f4e/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:14.754116 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab211fea-94bd-4693-9b4d-e12182020284 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:14.754844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab211fea-94bd-4693-9b4d-e12182020284 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:14.755291 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab211fea-94bd-4693-9b4d-e12182020284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:14.787456 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-ab211fea-94bd-4693-9b4d-e12182020284 None None] MFA Rules not processed for user `1ea733c4fd664a46bf4d959e8dac1f4e`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:14.788736 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-ab211fea-94bd-4693-9b4d-e12182020284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:14.867284 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1465/2924] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:31:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:31:14.878968 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba9ab82-448e-4578-871e-beb746f08a33 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:14.880916 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-2ba9ab82-448e-4578-871e-beb746f08a33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:14.893183 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:14.952381 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba9ab82-448e-4578-871e-beb746f08a33 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:14.956551 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05a919-9193-4eac-9f3a-930409241866 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d7d59080cd4832a45dbb6c0349cf4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:14.957213 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05a919-9193-4eac-9f3a-930409241866 service glance] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:14.957494 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05a919-9193-4eac-9f3a-930409241866 service glance] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:14.957763 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05a919-9193-4eac-9f3a-930409241866 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:14.967399 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd05a919-9193-4eac-9f3a-930409241866 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:14.968857 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd05a919-9193-4eac-9f3a-930409241866 service glance] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:14.978621 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1460/2925] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:31:24.863735 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:24.973931 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=23 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:33.653138 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab211fea-94bd-4693-9b4d-e12182020284 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:33.654817 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-ab211fea-94bd-4693-9b4d-e12182020284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:33.742267 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab211fea-94bd-4693-9b4d-e12182020284 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:33.749361 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6d0bd-a591-42e6-bbfd-d30bd79643b5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb857f32bbc04d82bfa681bd5659765f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:33.750029 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6d0bd-a591-42e6-bbfd-d30bd79643b5 service placement] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:33.750340 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6d0bd-a591-42e6-bbfd-d30bd79643b5 service placement] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:33.750921 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6d0bd-a591-42e6-bbfd-d30bd79643b5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:33.752984 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-20f6d0bd-a591-42e6-bbfd-d30bd79643b5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:33.765960 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6d0bd-a591-42e6-bbfd-d30bd79643b5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:33.767687 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6d0bd-a591-42e6-bbfd-d30bd79643b5 service placement] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:33.778754 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1466/2926] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:31:43.773928 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=32 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:53.783605 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05a919-9193-4eac-9f3a-930409241866 service glance] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:53.785152 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-bd05a919-9193-4eac-9f3a-930409241866 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:53.810124 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6d0bd-a591-42e6-bbfd-d30bd79643b5 service placement] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:53.862351 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05a919-9193-4eac-9f3a-930409241866 service glance] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:53.875032 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b353d-09ab-4af7-b714-03e94b51af12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:53.875991 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b353d-09ab-4af7-b714-03e94b51af12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:53.876285 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b353d-09ab-4af7-b714-03e94b51af12 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:53.876562 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b353d-09ab-4af7-b714-03e94b51af12 admin admin] PATH_INFO: `/v3/users/3776ccc4048b48dbbf6970975d5f054f` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:53.877910 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b353d-09ab-4af7-b714-03e94b51af12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3776ccc4048b48dbbf6970975d5f054f)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:53.879978 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b353d-09ab-4af7-b714-03e94b51af12 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:53.883127 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6d0bd-a591-42e6-bbfd-d30bd79643b5 service placement] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:53.889479 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9fdcf-aca4-4882-b771-d661f20061c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:53.890438 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9fdcf-aca4-4882-b771-d661f20061c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:53.890722 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9fdcf-aca4-4882-b771-d661f20061c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:53.891000 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9fdcf-aca4-4882-b771-d661f20061c6 admin admin] PATH_INFO: `/v3/users/1ea733c4fd664a46bf4d959e8dac1f4e` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:53.892344 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a9fdcf-aca4-4882-b771-d661f20061c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ea733c4fd664a46bf4d959e8dac1f4e)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:53.894306 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a9fdcf-aca4-4882-b771-d661f20061c6 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:53.934937 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-459b353d-09ab-4af7-b714-03e94b51af12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3776ccc4048b48dbbf6970975d5f054f'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:53.949237 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-459b353d-09ab-4af7-b714-03e94b51af12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3776ccc4048b48dbbf6970975d5f054f'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:53.949988 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-459b353d-09ab-4af7-b714-03e94b51af12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3776ccc4048b48dbbf6970975d5f054f'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:53.950628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-459b353d-09ab-4af7-b714-03e94b51af12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3776ccc4048b48dbbf6970975d5f054f'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:53.951407 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-f2a9fdcf-aca4-4882-b771-d661f20061c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ea733c4fd664a46bf4d959e8dac1f4e'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:53.957959 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-459b353d-09ab-4af7-b714-03e94b51af12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3776ccc4048b48dbbf6970975d5f054f'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:53.968688 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-f2a9fdcf-aca4-4882-b771-d661f20061c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ea733c4fd664a46bf4d959e8dac1f4e'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:53.969433 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-f2a9fdcf-aca4-4882-b771-d661f20061c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ea733c4fd664a46bf4d959e8dac1f4e'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:53.970044 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-f2a9fdcf-aca4-4882-b771-d661f20061c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ea733c4fd664a46bf4d959e8dac1f4e'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:53.974369 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1461/2927] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:31:53 2022] DELETE /identity/v3/users/3776ccc4048b48dbbf6970975d5f054f => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:53.978196 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-f2a9fdcf-aca4-4882-b771-d661f20061c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ea733c4fd664a46bf4d959e8dac1f4e'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:53.979302 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f561885-e109-427f-8ee8-ff4e93da2331 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:53.979637 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f561885-e109-427f-8ee8-ff4e93da2331 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:53.979859 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f561885-e109-427f-8ee8-ff4e93da2331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:53.992529 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1467/2928] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:31:53 2022] DELETE /identity/v3/users/1ea733c4fd664a46bf4d959e8dac1f4e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:53.997273 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f4998-a7ca-467d-b278-a9274d842695 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:53.997562 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f4998-a7ca-467d-b278-a9274d842695 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:53.997844 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f4998-a7ca-467d-b278-a9274d842695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:54.002035 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-6f561885-e109-427f-8ee8-ff4e93da2331 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:54.003458 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-6f561885-e109-427f-8ee8-ff4e93da2331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:54.016264 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-117f4998-a7ca-467d-b278-a9274d842695 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:54.017515 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-117f4998-a7ca-467d-b278-a9274d842695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:54.065708 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1462/2929] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:31:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:31:54.075857 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f561885-e109-427f-8ee8-ff4e93da2331 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:54.077429 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-6f561885-e109-427f-8ee8-ff4e93da2331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:54.083869 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1468/2930] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:31:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:31:54.094103 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f4998-a7ca-467d-b278-a9274d842695 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:54.096166 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-117f4998-a7ca-467d-b278-a9274d842695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:54.107131 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:54.119396 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:54.151557 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f561885-e109-427f-8ee8-ff4e93da2331 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:54.155461 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80811595-e6c4-439e-a542-0eaf8cb5a192 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:54.156142 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80811595-e6c4-439e-a542-0eaf8cb5a192 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:54.156425 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80811595-e6c4-439e-a542-0eaf8cb5a192 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:54.156707 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80811595-e6c4-439e-a542-0eaf8cb5a192 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:54.162851 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f4998-a7ca-467d-b278-a9274d842695 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:54.166745 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80811595-e6c4-439e-a542-0eaf8cb5a192 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:54.167188 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c15cef1-eaa1-4fc3-b29d-cb77674749fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:54.167864 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15cef1-eaa1-4fc3-b29d-cb77674749fe service neutron] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:54.168146 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15cef1-eaa1-4fc3-b29d-cb77674749fe service neutron] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:54.168301 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80811595-e6c4-439e-a542-0eaf8cb5a192 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:54.168422 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15cef1-eaa1-4fc3-b29d-cb77674749fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:54.178450 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c15cef1-eaa1-4fc3-b29d-cb77674749fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:54.178624 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1463/2931] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:31:54.179951 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c15cef1-eaa1-4fc3-b29d-cb77674749fe service neutron] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:54.190826 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1469/2932] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:31:54.554056 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80811595-e6c4-439e-a542-0eaf8cb5a192 service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:54.556558 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-80811595-e6c4-439e-a542-0eaf8cb5a192 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:54.582188 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80811595-e6c4-439e-a542-0eaf8cb5a192 service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:54.589366 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ffe99-e0c5-44fd-8f08-de7f1fbf8798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:54.590281 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ffe99-e0c5-44fd-8f08-de7f1fbf8798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:54.590562 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ffe99-e0c5-44fd-8f08-de7f1fbf8798 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:54.590888 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ffe99-e0c5-44fd-8f08-de7f1fbf8798 admin admin] PATH_INFO: `/v3/projects/16aba7fe10de4afa9c6b33ca31afbc5c` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:54.592142 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ffe99-e0c5-44fd-8f08-de7f1fbf8798 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16aba7fe10de4afa9c6b33ca31afbc5c)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:54.596998 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ffe99-e0c5-44fd-8f08-de7f1fbf8798 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:54.597556 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c15cef1-eaa1-4fc3-b29d-cb77674749fe service neutron] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:54.599513 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-7c15cef1-eaa1-4fc3-b29d-cb77674749fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:54.614664 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c15cef1-eaa1-4fc3-b29d-cb77674749fe service neutron] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:54.621364 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94466b-fdd7-4634-88a4-10ee9622edf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:54.623119 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94466b-fdd7-4634-88a4-10ee9622edf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:54.623427 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94466b-fdd7-4634-88a4-10ee9622edf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:54.623955 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94466b-fdd7-4634-88a4-10ee9622edf1 admin admin] PATH_INFO: `/v3/projects/051a8fff39f0432889e36fc60ab02702` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:54.625393 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb94466b-fdd7-4634-88a4-10ee9622edf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=051a8fff39f0432889e36fc60ab02702)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:54.627908 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb94466b-fdd7-4634-88a4-10ee9622edf1 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:54.652026 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ba5ffe99-e0c5-44fd-8f08-de7f1fbf8798 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16aba7fe10de4afa9c6b33ca31afbc5c', 'request_id': 'req-ba5ffe99-e0c5-44fd-8f08-de7f1fbf8798'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:54.657983 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ba5ffe99-e0c5-44fd-8f08-de7f1fbf8798 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16aba7fe10de4afa9c6b33ca31afbc5c', 'request_id': 'req-ba5ffe99-e0c5-44fd-8f08-de7f1fbf8798'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:54.666249 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ba5ffe99-e0c5-44fd-8f08-de7f1fbf8798 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16aba7fe10de4afa9c6b33ca31afbc5c', 'request_id': 'req-ba5ffe99-e0c5-44fd-8f08-de7f1fbf8798'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:54.679246 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-eb94466b-fdd7-4634-88a4-10ee9622edf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '051a8fff39f0432889e36fc60ab02702', 'request_id': 'req-eb94466b-fdd7-4634-88a4-10ee9622edf1'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:54.680422 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ba5ffe99-e0c5-44fd-8f08-de7f1fbf8798 admin admin] The token cache is being invalidate because project 16aba7fe10de4afa9c6b33ca31afbc5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86860) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:31:54.680719 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ba5ffe99-e0c5-44fd-8f08-de7f1fbf8798 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:54.681671 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-ba5ffe99-e0c5-44fd-8f08-de7f1fbf8798 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:54.683771 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1464/2933] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:31:54 2022] DELETE /identity/v3/projects/16aba7fe10de4afa9c6b33ca31afbc5c => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:54.684874 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-eb94466b-fdd7-4634-88a4-10ee9622edf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '051a8fff39f0432889e36fc60ab02702', 'request_id': 'req-eb94466b-fdd7-4634-88a4-10ee9622edf1'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:54.693257 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-eb94466b-fdd7-4634-88a4-10ee9622edf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '051a8fff39f0432889e36fc60ab02702', 'request_id': 'req-eb94466b-fdd7-4634-88a4-10ee9622edf1'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:54.707594 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-eb94466b-fdd7-4634-88a4-10ee9622edf1 admin admin] The token cache is being invalidate because project 051a8fff39f0432889e36fc60ab02702 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:31:54.707901 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-eb94466b-fdd7-4634-88a4-10ee9622edf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:54.708857 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-eb94466b-fdd7-4634-88a4-10ee9622edf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:54.710892 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1470/2934] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:31:54 2022] DELETE /identity/v3/projects/051a8fff39f0432889e36fc60ab02702 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:54.718233 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563bafe0-bec1-48e0-a9eb-296e64edf230 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:54.718498 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563bafe0-bec1-48e0-a9eb-296e64edf230 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:54.718733 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563bafe0-bec1-48e0-a9eb-296e64edf230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:54.739810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-563bafe0-bec1-48e0-a9eb-296e64edf230 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:54.741057 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-563bafe0-bec1-48e0-a9eb-296e64edf230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:54.811593 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1465/2935] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:31:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:54.821275 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318f779-f245-4639-a354-9abe1ed7236d None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:54.821805 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318f779-f245-4639-a354-9abe1ed7236d None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:54.822008 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318f779-f245-4639-a354-9abe1ed7236d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:54.849545 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-2318f779-f245-4639-a354-9abe1ed7236d None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:54.850742 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-2318f779-f245-4639-a354-9abe1ed7236d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:54.859642 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1471/2936] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:31:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:54.865025 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563bafe0-bec1-48e0-a9eb-296e64edf230 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:54.877578 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563bafe0-bec1-48e0-a9eb-296e64edf230 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:54.883097 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828136a1-f490-41ab-827c-4ce0de4bece8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:54.883819 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828136a1-f490-41ab-827c-4ce0de4bece8 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:54.884105 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828136a1-f490-41ab-827c-4ce0de4bece8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:54.884387 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828136a1-f490-41ab-827c-4ce0de4bece8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:54.885148 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828136a1-f490-41ab-827c-4ce0de4bece8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:31:54.885449 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828136a1-f490-41ab-827c-4ce0de4bece8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:54.887220 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828136a1-f490-41ab-827c-4ce0de4bece8 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:54.904629 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1466/2937] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 15:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:31:54.906213 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318f779-f245-4639-a354-9abe1ed7236d None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:54.917435 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318f779-f245-4639-a354-9abe1ed7236d None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:54.923555 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f3a59e-014c-47fd-bee4-56033259a708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:54.924337 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3a59e-014c-47fd-bee4-56033259a708 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:54.924685 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3a59e-014c-47fd-bee4-56033259a708 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:54.925022 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3a59e-014c-47fd-bee4-56033259a708 admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:54.925830 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f3a59e-014c-47fd-bee4-56033259a708 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:54.927975 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f3a59e-014c-47fd-bee4-56033259a708 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:54.962640 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1472/2938] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:31:54 2022] POST /identity/v3/projects => generated 369 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:31:54.964069 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828136a1-f490-41ab-827c-4ce0de4bece8 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:54.974628 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828136a1-f490-41ab-827c-4ce0de4bece8 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:55.008092 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:55.039235 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd2c32-3681-45ce-b9ca-faba30672426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:55.039918 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd2c32-3681-45ce-b9ca-faba30672426 admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:55.040202 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd2c32-3681-45ce-b9ca-faba30672426 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:55.040479 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd2c32-3681-45ce-b9ca-faba30672426 admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:55.041175 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd2c32-3681-45ce-b9ca-faba30672426 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:55.043108 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd2c32-3681-45ce-b9ca-faba30672426 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:55.067353 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1467/2939] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:31:54 2022] POST /identity/v3/users => generated 285 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:31:55.068636 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f3a59e-014c-47fd-bee4-56033259a708 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:55.079476 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f3a59e-014c-47fd-bee4-56033259a708 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:55.084718 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab2d37-dd59-4d78-85a0-ae566a78bd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:55.085381 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab2d37-dd59-4d78-85a0-ae566a78bd70 admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:55.085664 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab2d37-dd59-4d78-85a0-ae566a78bd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:55.085933 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab2d37-dd59-4d78-85a0-ae566a78bd70 admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:55.086656 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab2d37-dd59-4d78-85a0-ae566a78bd70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:55.088521 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab2d37-dd59-4d78-85a0-ae566a78bd70 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:55.099502 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1473/2940] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:31:55.105782 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd2c32-3681-45ce-b9ca-faba30672426 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:55.117505 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd2c32-3681-45ce-b9ca-faba30672426 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:55.124328 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd788127-fd2d-4d47-84c2-147a3630d063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:55.125013 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd788127-fd2d-4d47-84c2-147a3630d063 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:55.125295 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd788127-fd2d-4d47-84c2-147a3630d063 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:55.125568 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd788127-fd2d-4d47-84c2-147a3630d063 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:55.126398 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd788127-fd2d-4d47-84c2-147a3630d063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:55.128033 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd788127-fd2d-4d47-84c2-147a3630d063 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:55.139736 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1468/2941] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:31:55.145324 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab2d37-dd59-4d78-85a0-ae566a78bd70 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:55.160677 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab2d37-dd59-4d78-85a0-ae566a78bd70 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:55.166935 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa1e9d-1f0d-423e-84c1-e9c52186eb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:55.167978 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa1e9d-1f0d-423e-84c1-e9c52186eb59 admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:55.168284 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa1e9d-1f0d-423e-84c1-e9c52186eb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:55.168592 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa1e9d-1f0d-423e-84c1-e9c52186eb59 admin admin] PATH_INFO: `/v3/projects/29128b6c15ef4e8ba0ee631fc1b81a58/users/471e5422113f4a5b8733bea87fbf9590/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:55.187343 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afa1e9d-1f0d-423e-84c1-e9c52186eb59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29128b6c15ef4e8ba0ee631fc1b81a58, user_id=471e5422113f4a5b8733bea87fbf9590, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:55.194312 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afa1e9d-1f0d-423e-84c1-e9c52186eb59 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:55.220971 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1474/2942] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:31:55 2022] PUT /identity/v3/projects/29128b6c15ef4e8ba0ee631fc1b81a58/users/471e5422113f4a5b8733bea87fbf9590/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:55.236545 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8db98-4bbf-4b01-800d-498122d58c44 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:55.236875 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8db98-4bbf-4b01-800d-498122d58c44 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:55.237388 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8db98-4bbf-4b01-800d-498122d58c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:55.266491 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-0ed8db98-4bbf-4b01-800d-498122d58c44 None None] MFA Rules not processed for user `471e5422113f4a5b8733bea87fbf9590`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:55.267783 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-0ed8db98-4bbf-4b01-800d-498122d58c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:55.341427 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1469/2943] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:31:55 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:55.356567 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa1e9d-1f0d-423e-84c1-e9c52186eb59 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:55.358449 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-5afa1e9d-1f0d-423e-84c1-e9c52186eb59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:55.413566 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:55.431253 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa1e9d-1f0d-423e-84c1-e9c52186eb59 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:55.435275 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc6683-64d8-46ab-be0d-3a7a310b0c6a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d7d59080cd4832a45dbb6c0349cf4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:55.435923 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc6683-64d8-46ab-be0d-3a7a310b0c6a service glance] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:55.436192 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc6683-64d8-46ab-be0d-3a7a310b0c6a service glance] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:55.436466 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc6683-64d8-46ab-be0d-3a7a310b0c6a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:55.444944 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dc6683-64d8-46ab-be0d-3a7a310b0c6a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:55.446364 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dc6683-64d8-46ab-be0d-3a7a310b0c6a service glance] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:55.455399 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1475/2944] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:55 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:32:05.337931 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=51,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:05.451346 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=7 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:44.441476 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8db98-4bbf-4b01-800d-498122d58c44 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:44.512923 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8db98-4bbf-4b01-800d-498122d58c44 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:44.518893 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04012a6-d29c-4399-9370-b5469f1e623a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:44.519810 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04012a6-d29c-4399-9370-b5469f1e623a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:44.520091 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04012a6-d29c-4399-9370-b5469f1e623a admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:44.520364 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04012a6-d29c-4399-9370-b5469f1e623a admin admin] PATH_INFO: `/v3/users/471e5422113f4a5b8733bea87fbf9590` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:44.521600 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04012a6-d29c-4399-9370-b5469f1e623a admin admin] RBAC: Authorizing `identity:delete_user(user_id=471e5422113f4a5b8733bea87fbf9590)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:44.523634 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04012a6-d29c-4399-9370-b5469f1e623a admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:44.572379 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-e04012a6-d29c-4399-9370-b5469f1e623a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '471e5422113f4a5b8733bea87fbf9590'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:44.583992 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-e04012a6-d29c-4399-9370-b5469f1e623a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '471e5422113f4a5b8733bea87fbf9590'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:44.584670 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-e04012a6-d29c-4399-9370-b5469f1e623a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '471e5422113f4a5b8733bea87fbf9590'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:44.585278 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-e04012a6-d29c-4399-9370-b5469f1e623a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '471e5422113f4a5b8733bea87fbf9590'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:44.592216 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-e04012a6-d29c-4399-9370-b5469f1e623a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '471e5422113f4a5b8733bea87fbf9590'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:44.605891 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1470/2945] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:32:44 2022] DELETE /identity/v3/users/471e5422113f4a5b8733bea87fbf9590 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:44.610532 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f08d0-dd1f-4149-8d81-dcde6089a64e None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:44.611003 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f08d0-dd1f-4149-8d81-dcde6089a64e None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:44.611128 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f08d0-dd1f-4149-8d81-dcde6089a64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:44.629033 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-216f08d0-dd1f-4149-8d81-dcde6089a64e None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:44.630216 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-216f08d0-dd1f-4149-8d81-dcde6089a64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:44.691459 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1476/2946] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:32:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:44.705172 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04012a6-d29c-4399-9370-b5469f1e623a admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:44.707770 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-e04012a6-d29c-4399-9370-b5469f1e623a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:44.773247 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04012a6-d29c-4399-9370-b5469f1e623a admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:44.777407 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1338264-73d1-47e4-989c-1e2dab38a9f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:44.778085 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1338264-73d1-47e4-989c-1e2dab38a9f6 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:44.778391 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1338264-73d1-47e4-989c-1e2dab38a9f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:44.778694 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1338264-73d1-47e4-989c-1e2dab38a9f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:44.788737 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1338264-73d1-47e4-989c-1e2dab38a9f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:44.790227 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1338264-73d1-47e4-989c-1e2dab38a9f6 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:44.803811 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1471/2947] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:32:45.083807 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f08d0-dd1f-4149-8d81-dcde6089a64e None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:45.085301 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-216f08d0-dd1f-4149-8d81-dcde6089a64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:45.098652 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f08d0-dd1f-4149-8d81-dcde6089a64e None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:45.103915 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a4bbd9-7dff-4899-ae97-fb2face9516e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:45.104790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4bbd9-7dff-4899-ae97-fb2face9516e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:45.105073 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4bbd9-7dff-4899-ae97-fb2face9516e admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:45.105351 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4bbd9-7dff-4899-ae97-fb2face9516e admin admin] PATH_INFO: `/v3/projects/29128b6c15ef4e8ba0ee631fc1b81a58` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:45.106568 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a4bbd9-7dff-4899-ae97-fb2face9516e admin admin] RBAC: Authorizing `identity:delete_project(project_id=29128b6c15ef4e8ba0ee631fc1b81a58)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:45.108510 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a4bbd9-7dff-4899-ae97-fb2face9516e admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:45.159186 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a9a4bbd9-7dff-4899-ae97-fb2face9516e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29128b6c15ef4e8ba0ee631fc1b81a58', 'request_id': 'req-a9a4bbd9-7dff-4899-ae97-fb2face9516e'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:45.164693 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a9a4bbd9-7dff-4899-ae97-fb2face9516e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29128b6c15ef4e8ba0ee631fc1b81a58', 'request_id': 'req-a9a4bbd9-7dff-4899-ae97-fb2face9516e'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:45.174148 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a9a4bbd9-7dff-4899-ae97-fb2face9516e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29128b6c15ef4e8ba0ee631fc1b81a58', 'request_id': 'req-a9a4bbd9-7dff-4899-ae97-fb2face9516e'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:45.191016 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a9a4bbd9-7dff-4899-ae97-fb2face9516e admin admin] The token cache is being invalidate because project 29128b6c15ef4e8ba0ee631fc1b81a58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:32:45.191433 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a9a4bbd9-7dff-4899-ae97-fb2face9516e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:45.192128 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-a9a4bbd9-7dff-4899-ae97-fb2face9516e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:45.194482 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1477/2948] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:32:45 2022] DELETE /identity/v3/projects/29128b6c15ef4e8ba0ee631fc1b81a58 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:45.200019 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3049c58-e953-4511-b8d6-ef1cd4a82aef None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:45.200316 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3049c58-e953-4511-b8d6-ef1cd4a82aef None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:45.200645 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3049c58-e953-4511-b8d6-ef1cd4a82aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:45.220166 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-b3049c58-e953-4511-b8d6-ef1cd4a82aef None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:45.221398 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-b3049c58-e953-4511-b8d6-ef1cd4a82aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:45.240131 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:45.286358 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1472/2949] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:45.294763 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da3b0b-20ed-4b40-9f90-dfd31c41c2f1 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:45.295237 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da3b0b-20ed-4b40-9f90-dfd31c41c2f1 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:45.295522 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da3b0b-20ed-4b40-9f90-dfd31c41c2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:45.315289 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-65da3b0b-20ed-4b40-9f90-dfd31c41c2f1 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:45.316559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-65da3b0b-20ed-4b40-9f90-dfd31c41c2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:45.326702 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1478/2950] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:45.331881 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3049c58-e953-4511-b8d6-ef1cd4a82aef None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:45.344832 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3049c58-e953-4511-b8d6-ef1cd4a82aef None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:45.350426 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc3dd5-11ae-4c0d-8c48-66f981fde522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:45.351089 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc3dd5-11ae-4c0d-8c48-66f981fde522 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:45.351395 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc3dd5-11ae-4c0d-8c48-66f981fde522 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:45.351669 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc3dd5-11ae-4c0d-8c48-66f981fde522 admin admin] PATH_INFO: `/v3/domains` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:45.352401 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc3dd5-11ae-4c0d-8c48-66f981fde522 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86860) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:32:45.353073 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc3dd5-11ae-4c0d-8c48-66f981fde522 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:45.354670 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc3dd5-11ae-4c0d-8c48-66f981fde522 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:45.367227 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1473/2951] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jul 20 15:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:32:45.371994 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da3b0b-20ed-4b40-9f90-dfd31c41c2f1 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:45.381320 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da3b0b-20ed-4b40-9f90-dfd31c41c2f1 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:45.386675 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b78edf-d2e7-4c59-9aa7-4e8aab957240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:45.387377 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b78edf-d2e7-4c59-9aa7-4e8aab957240 admin admin] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:45.387643 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b78edf-d2e7-4c59-9aa7-4e8aab957240 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:45.387913 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b78edf-d2e7-4c59-9aa7-4e8aab957240 admin admin] PATH_INFO: `/v3/projects` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:45.388610 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b78edf-d2e7-4c59-9aa7-4e8aab957240 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:45.390503 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b78edf-d2e7-4c59-9aa7-4e8aab957240 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:45.418341 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1479/2952] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jul 20 15:32:45 2022] POST /identity/v3/projects => generated 397 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:45.424801 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc3dd5-11ae-4c0d-8c48-66f981fde522 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:45.438690 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc3dd5-11ae-4c0d-8c48-66f981fde522 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:45.507620 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb37d3f-eedc-44f3-b948-ac03948618fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:45.508318 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb37d3f-eedc-44f3-b948-ac03948618fd admin admin] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:45.508613 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb37d3f-eedc-44f3-b948-ac03948618fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:45.508890 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb37d3f-eedc-44f3-b948-ac03948618fd admin admin] PATH_INFO: `/v3/users` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:45.509625 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb37d3f-eedc-44f3-b948-ac03948618fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:45.511681 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb37d3f-eedc-44f3-b948-ac03948618fd admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:45.534161 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1474/2953] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jul 20 15:32:45 2022] POST /identity/v3/users => generated 299 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:45.538790 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b78edf-d2e7-4c59-9aa7-4e8aab957240 admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:45.549930 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b78edf-d2e7-4c59-9aa7-4e8aab957240 admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:45.554837 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4931b52a-1afe-426e-9985-4919a2ffcfad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:45.555550 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931b52a-1afe-426e-9985-4919a2ffcfad admin admin] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:45.555816 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931b52a-1afe-426e-9985-4919a2ffcfad admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:45.556214 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931b52a-1afe-426e-9985-4919a2ffcfad admin admin] PATH_INFO: `/v3/roles` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:45.556951 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4931b52a-1afe-426e-9985-4919a2ffcfad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:45.558558 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4931b52a-1afe-426e-9985-4919a2ffcfad admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:45.569540 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1480/2954] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:45.576297 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb37d3f-eedc-44f3-b948-ac03948618fd admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:45.591566 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb37d3f-eedc-44f3-b948-ac03948618fd admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:45.597286 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f480726-72cb-4921-a7dc-dd83466fa9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:45.597965 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f480726-72cb-4921-a7dc-dd83466fa9c3 admin admin] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:45.598255 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f480726-72cb-4921-a7dc-dd83466fa9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:45.598539 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f480726-72cb-4921-a7dc-dd83466fa9c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:45.599313 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f480726-72cb-4921-a7dc-dd83466fa9c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:45.600931 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f480726-72cb-4921-a7dc-dd83466fa9c3 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:45.612270 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1475/2955] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:32:45.618977 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4931b52a-1afe-426e-9985-4919a2ffcfad admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:45.629604 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4931b52a-1afe-426e-9985-4919a2ffcfad admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:45.635780 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571864ef-86bb-4d6c-9736-be4f066ce1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:45.636674 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571864ef-86bb-4d6c-9736-be4f066ce1ed admin admin] REQUEST_METHOD: `PUT` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:45.636963 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571864ef-86bb-4d6c-9736-be4f066ce1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:45.637242 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571864ef-86bb-4d6c-9736-be4f066ce1ed admin admin] PATH_INFO: `/v3/projects/5e08998a80c840669ffbbeed99a02ab8/users/083020f6999c4b568653d080280ee56d/roles/7e394dd183b147ffa15b617f076bbb76` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:45.654114 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571864ef-86bb-4d6c-9736-be4f066ce1ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e08998a80c840669ffbbeed99a02ab8, user_id=083020f6999c4b568653d080280ee56d, role_id=7e394dd183b147ffa15b617f076bbb76)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:45.656849 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571864ef-86bb-4d6c-9736-be4f066ce1ed admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:45.672576 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1481/2956] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jul 20 15:32:45 2022] PUT /identity/v3/projects/5e08998a80c840669ffbbeed99a02ab8/users/083020f6999c4b568653d080280ee56d/roles/7e394dd183b147ffa15b617f076bbb76 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:45.683882 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc3f0d-e317-43ac-adce-b7fa16fed538 None None] REQUEST_METHOD: `POST` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:45.684173 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc3f0d-e317-43ac-adce-b7fa16fed538 None None] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:45.684451 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc3f0d-e317-43ac-adce-b7fa16fed538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:45.711750 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.auth.core [None req-7efc3f0d-e317-43ac-adce-b7fa16fed538 None None] MFA Rules not processed for user `083020f6999c4b568653d080280ee56d`. Rule list: `[]` (Enabled: `True`). {{(pid=86860) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:45.713091 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-7efc3f0d-e317-43ac-adce-b7fa16fed538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:45.796807 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1476/2957] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:32:45 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:45.810948 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571864ef-86bb-4d6c-9736-be4f066ce1ed admin admin] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:45.813032 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-571864ef-86bb-4d6c-9736-be4f066ce1ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:45.838438 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:45.878719 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571864ef-86bb-4d6c-9736-be4f066ce1ed admin admin] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:45.882704 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a66133-3fc5-411d-890e-7eae6f2a62db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9618ec9bbe76431e84640b0e5ca90eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:45.883385 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a66133-3fc5-411d-890e-7eae6f2a62db service cinder] REQUEST_METHOD: `GET` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:45.883668 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a66133-3fc5-411d-890e-7eae6f2a62db service cinder] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:45.883967 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a66133-3fc5-411d-890e-7eae6f2a62db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:45.892538 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a66133-3fc5-411d-890e-7eae6f2a62db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:45.893964 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a66133-3fc5-411d-890e-7eae6f2a62db service cinder] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:45.903077 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1482/2958] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:45 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:32:55.793496 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:55.899088 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=17 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:57.134953 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc3f0d-e317-43ac-adce-b7fa16fed538 None None] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:57.207298 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc3f0d-e317-43ac-adce-b7fa16fed538 None None] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:57.213076 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75078de-1c10-447c-9751-d64fee468b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:57.213983 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75078de-1c10-447c-9751-d64fee468b24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:57.214273 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75078de-1c10-447c-9751-d64fee468b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:57.214557 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75078de-1c10-447c-9751-d64fee468b24 admin admin] PATH_INFO: `/v3/users/083020f6999c4b568653d080280ee56d` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:57.215752 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75078de-1c10-447c-9751-d64fee468b24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=083020f6999c4b568653d080280ee56d)` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:57.217780 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75078de-1c10-447c-9751-d64fee468b24 admin admin] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:57.274471 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-b75078de-1c10-447c-9751-d64fee468b24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '083020f6999c4b568653d080280ee56d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:57.287582 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-b75078de-1c10-447c-9751-d64fee468b24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '083020f6999c4b568653d080280ee56d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:57.288041 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-b75078de-1c10-447c-9751-d64fee468b24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '083020f6999c4b568653d080280ee56d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:57.288707 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-b75078de-1c10-447c-9751-d64fee468b24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '083020f6999c4b568653d080280ee56d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:57.296520 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.notifications [None req-b75078de-1c10-447c-9751-d64fee468b24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '083020f6999c4b568653d080280ee56d'} {{(pid=86860) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:57.310959 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1477/2959] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jul 20 15:32:57 2022] DELETE /identity/v3/users/083020f6999c4b568653d080280ee56d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:57.315847 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6d5d3-5eb3-4523-9068-5fb10733b617 None None] REQUEST_METHOD: `POST` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:57.316298 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6d5d3-5eb3-4523-9068-5fb10733b617 None None] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:57.316491 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6d5d3-5eb3-4523-9068-5fb10733b617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:57.335069 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.auth.core [None req-1ae6d5d3-5eb3-4523-9068-5fb10733b617 None None] MFA Rules not processed for user `651b2280231048d59d156e569aa33194`. Rule list: `[]` (Enabled: `True`). {{(pid=86861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:57.336309 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-1ae6d5d3-5eb3-4523-9068-5fb10733b617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:57.403414 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1483/2960] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jul 20 15:32:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:57.415099 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75078de-1c10-447c-9751-d64fee468b24 admin admin] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:57.416985 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-b75078de-1c10-447c-9751-d64fee468b24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:57.489762 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75078de-1c10-447c-9751-d64fee468b24 admin admin] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:57.493854 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d95d6-bbaf-4810-9293-62cac8211226 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7658ffde740d446682c1876662815987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:57.494512 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d95d6-bbaf-4810-9293-62cac8211226 service neutron] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:57.494799 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d95d6-bbaf-4810-9293-62cac8211226 service neutron] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:57.495088 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d95d6-bbaf-4810-9293-62cac8211226 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:57.504716 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d95d6-bbaf-4810-9293-62cac8211226 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:57.506144 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d95d6-bbaf-4810-9293-62cac8211226 service neutron] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:57.515688 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1478/2961] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:32:57.805598 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae6d5d3-5eb3-4523-9068-5fb10733b617 None None] Authenticating user token {{(pid=86861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:57.807464 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.fernet_utils [None req-1ae6d5d3-5eb3-4523-9068-5fb10733b617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:57.820497 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae6d5d3-5eb3-4523-9068-5fb10733b617 None None] Validating token access rules against request {{(pid=86861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:57.826747 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aeb423-4153-4e9d-96b3-1de2d54695d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '651b2280231048d59d156e569aa33194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ca83facd1c8420dacb9e91e8abe0fd4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:57.827665 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aeb423-4153-4e9d-96b3-1de2d54695d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:57.827933 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aeb423-4153-4e9d-96b3-1de2d54695d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:57.828209 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aeb423-4153-4e9d-96b3-1de2d54695d4 admin admin] PATH_INFO: `/v3/projects/5e08998a80c840669ffbbeed99a02ab8` {{(pid=86861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:57.829559 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aeb423-4153-4e9d-96b3-1de2d54695d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e08998a80c840669ffbbeed99a02ab8)` {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:57.831535 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aeb423-4153-4e9d-96b3-1de2d54695d4 admin admin] RBAC: Authorization granted {{(pid=86861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:57.888360 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-99aeb423-4153-4e9d-96b3-1de2d54695d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e08998a80c840669ffbbeed99a02ab8', 'request_id': 'req-99aeb423-4153-4e9d-96b3-1de2d54695d4'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:57.901529 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-99aeb423-4153-4e9d-96b3-1de2d54695d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e08998a80c840669ffbbeed99a02ab8', 'request_id': 'req-99aeb423-4153-4e9d-96b3-1de2d54695d4'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:57.909793 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-99aeb423-4153-4e9d-96b3-1de2d54695d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e08998a80c840669ffbbeed99a02ab8', 'request_id': 'req-99aeb423-4153-4e9d-96b3-1de2d54695d4'} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:57.921837 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-99aeb423-4153-4e9d-96b3-1de2d54695d4 admin admin] The token cache is being invalidate because project 5e08998a80c840669ffbbeed99a02ab8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:32:57.922096 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-99aeb423-4153-4e9d-96b3-1de2d54695d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:57.922757 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG keystone.notifications [None req-99aeb423-4153-4e9d-96b3-1de2d54695d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:57.924371 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: [pid: 86861|app: 0|req: 1484/2962] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jul 20 15:32:57 2022] DELETE /identity/v3/projects/5e08998a80c840669ffbbeed99a02ab8 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:33:07.511480 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:07.918845 dv-cinder-849022-4-iscsi devstack@keystone.service[86861]: DEBUG dbcounter [-] [86861] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:40.725990 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d95d6-bbaf-4810-9293-62cac8211226 service neutron] Authenticating user token {{(pid=86860) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:40.727652 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-381d95d6-bbaf-4810-9293-62cac8211226 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:40.865949 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d95d6-bbaf-4810-9293-62cac8211226 service neutron] Validating token access rules against request {{(pid=86860) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:40.958116 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35712df9-c6e5-4c51-9553-12ed28c2fda8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb857f32bbc04d82bfa681bd5659765f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59333e4f593346e184877d6bd9d22026', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86860) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:40.958927 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35712df9-c6e5-4c51-9553-12ed28c2fda8 service placement] REQUEST_METHOD: `GET` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:40.959327 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35712df9-c6e5-4c51-9553-12ed28c2fda8 service placement] SCRIPT_NAME: `/identity` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:40.959621 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35712df9-c6e5-4c51-9553-12ed28c2fda8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86860) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:40.962352 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.fernet_utils [None req-35712df9-c6e5-4c51-9553-12ed28c2fda8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86860) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:40.981154 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35712df9-c6e5-4c51-9553-12ed28c2fda8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:40.983740 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35712df9-c6e5-4c51-9553-12ed28c2fda8 service placement] RBAC: Authorization granted {{(pid=86860) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:40.995700 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: [pid: 86860|app: 0|req: 1479/2963] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:36:50.990553 dv-cinder-849022-4-iscsi devstack@keystone.service[86860]: DEBUG dbcounter [-] [86860] Writing DB stats keystone:SELECT=76 {{(pid=86860) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}